Jul 14 00:57:55.861114 ip-10-0-141-32 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 14 00:57:55.861123 ip-10-0-141-32 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 14 00:57:55.861130 ip-10-0-141-32 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 14 00:57:55.861356 ip-10-0-141-32 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 14 00:58:07.074407 ip-10-0-141-32 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 14 00:58:07.074426 ip-10-0-141-32 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8e892bc96432488a8d457b1167319594 --
Jul 14 01:00:08.987804 ip-10-0-141-32 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 01:00:09.380185 ip-10-0-141-32 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 01:00:09.380185 ip-10-0-141-32 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 01:00:09.380185 ip-10-0-141-32 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 01:00:09.380185 ip-10-0-141-32 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 01:00:09.380805 ip-10-0-141-32 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 01:00:09.382808 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.382697    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 01:00:09.389759 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389738    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 01:00:09.389759 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389756    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 01:00:09.389759 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389761    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 01:00:09.389759 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389764    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 01:00:09.389759 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389767    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389770    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389773    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389776    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389779    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389782    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389784    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389787    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389789    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389792    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389795    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389797    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389800    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389802    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389805    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389808    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389810    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389813    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389815    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 01:00:09.389947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389818    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389820    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389824    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389829    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389832    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389835    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389838    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389841    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389845    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389848    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389850    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389853    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389856    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389860    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389863    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389867    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389870    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389873    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389876    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 01:00:09.390451 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389878    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389881    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389884    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389887    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389889    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389892    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389895    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389897    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389901    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389904    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389907    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389909    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389912    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389914    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389917    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389920    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389922    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389925    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389927    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 01:00:09.390940 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389930    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389934    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389936    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389939    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389941    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389944    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389946    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389949    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389952    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389955    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389958    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389961    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389963    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389966    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389969    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389971    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389974    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389976    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389979    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389981    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 01:00:09.391417 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389990    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389994    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.389997    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390001    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390004    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390007    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390010    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390012    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390015    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390017    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390020    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390022    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390025    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390028    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390031    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390034    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390037    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390039    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390758    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390765    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 01:00:09.391958 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390768    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390772    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390774    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390777    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390780    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390782    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390785    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390789    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390791    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390794    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390796    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390799    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390802    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390804    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390807    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390811    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390813    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390816    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390818    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390821    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 01:00:09.392440 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390823    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390826    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390829    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390832    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390834    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390837    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390840    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390843    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390845    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390848    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390851    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390853    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390856    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390859    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390861    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390866    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390870    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390873    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390876    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 01:00:09.392951 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390879    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390882    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390884    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390887    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390890    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390892    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390895    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390897    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390900    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390904    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390907    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390910    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390912    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390915    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390917    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390920    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390923    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390926    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390929    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390931    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 01:00:09.393428 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390934    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390937    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390939    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390942    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390945    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390948    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390951    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390954    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390957    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390960    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390964    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390966    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390969    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390972    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390975    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390977    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390981    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390985    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390988    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 01:00:09.394034 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390991    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390994    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.390996    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391000    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391003    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391006    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391009    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391011    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391014    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391017    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391020    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391023    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391026    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391029    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391031    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391034    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391037    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391040    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391042    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391465    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 01:00:09.394487 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391472    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391475    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391478    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391481    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391484    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391487    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391490    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391493    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391496    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391499    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391502    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391505    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391507    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391509    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391512    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391515    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391533    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391536    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391539    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391541    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 01:00:09.395024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391544    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391547    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391549    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391552    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391555    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391557    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391560    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391563    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391566    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391569    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391571    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391574    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391576    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391578    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391581    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391584    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391586    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391589    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391591    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391594    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 01:00:09.395600 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391597    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391599    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391603    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391607    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391611    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391614    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391617    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391619    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391626    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391628    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391631    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391635    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391638    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391640    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391643    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391646    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391648    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391651    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391653    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 01:00:09.396086 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391655    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391658    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391661    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391663    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391666    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391668    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391671    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391674    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391676    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391678    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391682    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391684    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391687    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391689    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391692    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391694    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391697    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391699    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391702    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391704    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 01:00:09.396561 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391707    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391710    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391713    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391715    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391718    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391720    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391723    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391725    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391728    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391730    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391733    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391735    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391738    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391741    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391743    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391747    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391751    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391753    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.391757    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391820    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 01:00:09.397081 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391827    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391832    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391837    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391840    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391843    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391847    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391850    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391853    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391856    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391860    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391863    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391866    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391869    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391872    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391875    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391878    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391881    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391886    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391888    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391891    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391894    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391897    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391901    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 01:00:09.397579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391904    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391907    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391910    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391913    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391916    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391918    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391922    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391924    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391928    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391931    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391934    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391937    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391940    2580 flags.go:64] FLAG: --enable-server="true"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391942    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391946    2580 flags.go:64] FLAG: --event-burst="100"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391949    2580 flags.go:64] FLAG: --event-qps="50"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391952    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391955    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391958    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391964    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391966    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391969    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391973    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391975    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391978    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 01:00:09.398166 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391982    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391985    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391987    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391990    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391993    2580 flags.go:64] FLAG: --feature-gates=""
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391996    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.391999    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392002    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392005    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392010    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392012    2580 flags.go:64] FLAG: --help="false"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392015    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392027    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392030    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392033    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392036    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392040    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392042    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392045    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392048    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392051    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392053    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392056    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392059    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 14 01:00:09.398799 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392062    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392065    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392070    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392073    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392076    2580 flags.go:64] FLAG: --lock-file=""
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392079    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392082    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392085    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392090    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392093    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392096    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392099    2580 flags.go:64] FLAG: --logging-format="text"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392101    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392105    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392107    2580 flags.go:64] FLAG: --manifest-url=""
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392110    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392114    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392119    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392123    2580 flags.go:64] FLAG: --max-pods="110"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392125    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392129    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392132    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392135    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392138    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392141    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 01:00:09.399386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392143    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392150    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392153    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392156    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392159    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392162    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392167    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392169    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392172    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392175    2580 flags.go:64] FLAG: --port="10250"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392179    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392182    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04af123e476efd0d0"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392185    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392188    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392190    2580 flags.go:64] FLAG: --register-node="true"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392193    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392197    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392200    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392203    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392205    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392209    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392212    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392215    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392217    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392222    2580 flags.go:64] FLAG: --runonce="false"
Jul 14 01:00:09.400021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392225    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392228    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392231    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392233    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392236    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392239    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392242    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392245    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392248    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392250    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392253    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392256    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392259    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392261    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392264    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392269    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392272    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392276    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392284    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392287    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392289    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392292    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392295    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392298    2580 flags.go:64] FLAG: --v="2"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392302    2580 flags.go:64] FLAG: --version="false"
Jul 14 01:00:09.400678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392307    2580 flags.go:64] FLAG: --vmodule=""
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392310    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392313    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392417    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392421    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392424    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392429    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392431    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392434    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392437    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392439    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392442    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392445    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392448    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392451    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392453    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392456    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392458    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392461    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392464    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 01:00:09.401278 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392466    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392469    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392471    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392474    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392478    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392480    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392483    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392486    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392488    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392491    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392493    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392496    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392498    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392501    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392504    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392506    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392509    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392512    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392515    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392533    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 01:00:09.401791 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392537    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392540    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392544    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392547    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392550    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392552    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392555    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392558    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392561    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392563    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392566    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392569    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392571    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392574    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392576    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392579    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392582    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392585    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392588    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392590    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 01:00:09.402355 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392593    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392595    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392598    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392600    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392603    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392606    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392609    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392611    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392614    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392616    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392620    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392623    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392625    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392628    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392630    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392633    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392635    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392638    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392640    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392643    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 01:00:09.402870 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392646    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392649    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392651    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392654    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392656    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392659    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392663    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392666    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392670    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392672    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392675    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392679    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392683    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392686    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392689    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392691    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392694    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392696    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 01:00:09.403366 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392699    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392702    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392704    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.392707    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.392716    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.399090    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.399104    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.399122    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.399128    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399185    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399190    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399194    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399198    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399201    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399204    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 01:00:09.404048 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399207    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399210    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399213    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399215    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399218    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399220    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399223    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399225    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399228    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399231    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399233    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399236    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399238    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399241    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399243    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399246    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399249    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399251    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399255    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 01:00:09.404741 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399258    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399262    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399265    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399267    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399270    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399273    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399276    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399278    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399281    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399283    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399286    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399288    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399291    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399294    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399296    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399299    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399301    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399304    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399306    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399309    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 01:00:09.405229 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399311    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399313    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399316    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399318    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399321    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399323    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399326    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399328    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399331    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399334    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399336    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399339    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399346    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399349    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399351    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399354    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399357    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399360    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399363    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 01:00:09.405900 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399365    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399368    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399370    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399373    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399375    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399379    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399383    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399386    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399389    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399392    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399395    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399399    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399402    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399405    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399408    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399411    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399413    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399416    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399419    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 01:00:09.406362 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399421    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399424    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399427    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399429    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399432    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399435    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399438    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399441    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399444    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399446    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399449    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399452    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399454    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399457    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399460    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 01:00:09.407160 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399462    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.399468    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399612    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399618    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399621    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399624    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399627    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399630    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399633    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399636    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399638    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399641    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399644    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399647    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399649    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 01:00:09.407657 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399652    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399654    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399657    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399659    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399662    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399664    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399667    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399669    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399672    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399674    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399678    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399680    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399683    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399686    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399689    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399692    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399696    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399699    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399702    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 01:00:09.408033 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399705    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399708    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399710    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399713    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399716    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399718    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399721    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399723    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399726    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399728    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399731    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399733    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399736    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399738    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399740    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399743    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399747    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399750    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399752    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 01:00:09.408479 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399755    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399758    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399760    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399763    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399765    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399773    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399776    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399780    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399783    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399785    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399788    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399790    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399793    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399795    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399797    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399800    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399802    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399805    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399807    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 01:00:09.408970 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399810    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399812    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399814    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399817    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399819    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399822    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399824    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399826    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399829    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399831    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399834    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399836    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399839    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399842    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399844    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399847    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399849    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399852    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399855    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399862    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 01:00:09.409483 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399865    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399867    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399870    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399872    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399875    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399877    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399880    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399882    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:09.399884    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.399890    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.399981    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.401976    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.403409    2580 server.go:1019] "Starting client certificate rotation"
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.403556    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 01:00:09.409986 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.403612    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 01:00:09.410333 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.405720    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 01:00:09.410333 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.409645    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 01:00:09.420180 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.420161    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 14 01:00:09.422006 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.421990    2580 log.go:25] "Validated CRI v1 image API"
Jul 14 01:00:09.422756 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.422739    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 01:00:09.427108 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.427083    2580 fs.go:135] Filesystem UUIDs: map[138e4a35-df10-422d-b1b6-179ae93964d2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c32969a3-7650-4a1f-918f-ad108c06ccda:/dev/nvme0n1p4]
Jul 14 01:00:09.427184 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.427107    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 01:00:09.427455 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.427433    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 01:00:09.432000 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.431889    2580 manager.go:216] Machine: {Timestamp:2026-07-14 01:00:09.430643819 +0000 UTC m=+0.325494449 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096666 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24e6c03219917743c272f0946bfa51 SystemUUID:ec24e6c0-3219-9177-43c2-72f0946bfa51 BootID:8e892bc9-6432-488a-8d45-7b1167319594 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c6:22:0d:be:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c6:22:0d:be:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:1b:a2:4a:c8:3f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 01:00:09.432000 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.431992    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 01:00:09.432164 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.432125    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 01:00:09.434605 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.434584    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 01:00:09.434735 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.434607    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-32.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 01:00:09.435296 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.435286    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 01:00:09.435330 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.435299    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 01:00:09.435330 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.435306    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 01:00:09.435900 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.435889    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 01:00:09.437033 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.437023    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 01:00:09.440882 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.440870    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 01:00:09.441005 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.440995    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 01:00:09.441059 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.441024    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 01:00:09.441059 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.441034    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 01:00:09.441144 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.441061    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 01:00:09.441144 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.441122    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 14 01:00:09.441144 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.441131    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 01:00:09.441418 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.441408    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 01:00:09.441451 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.441424    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 01:00:09.443659 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.443637    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 01:00:09.445659 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.445646    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 01:00:09.445708 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.445677    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 01:00:09.445824 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.445815    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 01:00:09.445853 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.445829    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 01:00:09.445853 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.445835    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 01:00:09.445853 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.445840    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 01:00:09.445853 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.445848    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 01:00:09.445853 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.445854    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 01:00:09.446419 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.446411    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 01:00:09.446450 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.446419    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 01:00:09.446450 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.446427    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 01:00:09.446450 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.446432    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 01:00:09.446450 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.446440    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 01:00:09.446450 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.446448    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 01:00:09.448055 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.448044    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 01:00:09.448091 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.448057    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 01:00:09.451023 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.450997    2580 server.go:1268] "Started kubelet"
Jul 14 01:00:09.451127 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.451097    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 01:00:09.451177 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.451155    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 01:00:09.451469 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.451438    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 01:00:09.451876 ip-10-0-141-32 systemd[1]: Started Kubernetes Kubelet.
Jul 14 01:00:09.452506 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.452472    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 01:00:09.455442 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.455373    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-32.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 01:00:09.455740 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.455701    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 01:00:09.455812 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.455735    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-32.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 01:00:09.455876 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.455850    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xl68h"
Jul 14 01:00:09.455876 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.455858    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 01:00:09.457341 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.457320    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 01:00:09.458409 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.457505    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-32.ec2.internal.18c202306fe2f969  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-32.ec2.internal,UID:ip-10-0-141-32.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-32.ec2.internal,},FirstTimestamp:2026-07-14 01:00:09.450969449 +0000 UTC m=+0.345820079,LastTimestamp:2026-07-14 01:00:09.450969449 +0000 UTC m=+0.345820079,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-32.ec2.internal,}"
Jul 14 01:00:09.459671 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.459654    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 01:00:09.459671 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.459661    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 01:00:09.459797 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.459754    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:09.460283 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.460265    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 01:00:09.460369 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.460309    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 01:00:09.460369 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.460324    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 01:00:09.460459 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.460413    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 01:00:09.460459 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.460422    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 01:00:09.460459 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.460448    2580 factory.go:55] Registering systemd factory
Jul 14 01:00:09.460615 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.460483    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 14 01:00:09.462060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.461821    2580 factory.go:153] Registering CRI-O factory
Jul 14 01:00:09.462060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.461838    2580 factory.go:223] Registration of the crio container factory successfully
Jul 14 01:00:09.462060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.461892    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 01:00:09.462060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.461914    2580 factory.go:103] Registering Raw factory
Jul 14 01:00:09.462060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.461929    2580 manager.go:1196] Started watching for new ooms in manager
Jul 14 01:00:09.462060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.461942    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xl68h"
Jul 14 01:00:09.462547 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.462312    2580 manager.go:349] Starting recovery of all containers
Jul 14 01:00:09.464589 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.464562    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 01:00:09.464685 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.464661    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-32.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 01:00:09.473879 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.473868    2580 manager.go:354] Recovery completed
Jul 14 01:00:09.477939 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.477927    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 01:00:09.480023 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.480001    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 01:00:09.480076 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.480027    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 01:00:09.480076 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.480037    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientPID"
Jul 14 01:00:09.480566 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.480504    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 01:00:09.480566 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.480533    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 01:00:09.480566 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.480550    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 01:00:09.480705 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.480514    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 01:00:09.481317 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.481305    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 01:00:09.481360 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.481329    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 01:00:09.481360 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.481339    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientPID"
Jul 14 01:00:09.482666 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.482653    2580 policy_none.go:49] "None policy: Start"
Jul 14 01:00:09.482740 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.482671    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 01:00:09.482740 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.482684    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 01:00:09.484725 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.484711    2580 policy_none.go:47] "Start"
Jul 14 01:00:09.519121 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.519102    2580 manager.go:337] "Starting Device Plugin manager"
Jul 14 01:00:09.519202 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.519150    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 01:00:09.519202 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.519163    2580 server.go:87] "Starting device plugin registration server"
Jul 14 01:00:09.519394 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.519380    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 01:00:09.519452 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.519396    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 01:00:09.519498 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.519469    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 01:00:09.519575 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.519566    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 01:00:09.519575 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.519574    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 01:00:09.520081 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.520060    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 01:00:09.520172 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.520109    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:09.619935 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.619903    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 01:00:09.621580 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.621563    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 01:00:09.621641 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.621594    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 01:00:09.621641 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.621605    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientPID"
Jul 14 01:00:09.621641 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.621630    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.627460 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.627446    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.627512 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.627467    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-32.ec2.internal\": node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:09.634737 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.634683    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 01:00:09.635945 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.635927    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 01:00:09.636003 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.635976    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 01:00:09.636003 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.635999    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 01:00:09.636089 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.636076    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 01:00:09.638582 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.638565    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 01:00:09.641136 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.641120    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:09.736149 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.736114    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal"]
Jul 14 01:00:09.736232 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.736182    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 01:00:09.737497 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.737482    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 01:00:09.737586 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.737509    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 01:00:09.737586 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.737548    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientPID"
Jul 14 01:00:09.739457 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.739445    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 01:00:09.739601 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.739587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.739661 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.739614    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 01:00:09.740465 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.740450    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 01:00:09.740550 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.740450    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 01:00:09.740550 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.740477    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 01:00:09.740550 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.740489    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientPID"
Jul 14 01:00:09.740550 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.740498    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 01:00:09.740550 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.740510    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientPID"
Jul 14 01:00:09.741166 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.741154    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:09.742252 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.742233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.742344 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.742261    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 01:00:09.742947 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.742931    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 01:00:09.743039 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.742957    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 01:00:09.743039 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.742967    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeHasSufficientPID"
Jul 14 01:00:09.761858 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.761832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ad49a4c43a161f6386858ac515eb52ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal\" (UID: \"ad49a4c43a161f6386858ac515eb52ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.761944 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.761859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad49a4c43a161f6386858ac515eb52ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal\" (UID: \"ad49a4c43a161f6386858ac515eb52ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.761944 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.761875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d3082df98acf7e090e04a68eb61aa6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-32.ec2.internal\" (UID: \"9d3082df98acf7e090e04a68eb61aa6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.763083 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.763059    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-32.ec2.internal\" not found" node="ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.767440 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.767427    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-32.ec2.internal\" not found" node="ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.841952 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.841931    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:09.862302 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.862277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ad49a4c43a161f6386858ac515eb52ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal\" (UID: \"ad49a4c43a161f6386858ac515eb52ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.862302 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.862303    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad49a4c43a161f6386858ac515eb52ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal\" (UID: \"ad49a4c43a161f6386858ac515eb52ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.862421 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.862319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d3082df98acf7e090e04a68eb61aa6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-32.ec2.internal\" (UID: \"9d3082df98acf7e090e04a68eb61aa6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.862421 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.862368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d3082df98acf7e090e04a68eb61aa6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-32.ec2.internal\" (UID: \"9d3082df98acf7e090e04a68eb61aa6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.862421 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.862376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ad49a4c43a161f6386858ac515eb52ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal\" (UID: \"ad49a4c43a161f6386858ac515eb52ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.862421 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:09.862396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad49a4c43a161f6386858ac515eb52ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal\" (UID: \"ad49a4c43a161f6386858ac515eb52ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:09.942608 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:09.942549    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.043055 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.043025    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.065565 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.065548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:10.070148 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.069960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:10.144670 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.144640    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.250980 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.250925    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.351113 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.351078    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.404490 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.404465    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 01:00:10.405068 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.404629    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 01:00:10.446881 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.446859    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 01:00:10.451579 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.451555    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.466666 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.466637    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 00:55:09 +0000 UTC" deadline="2028-03-07 05:48:20.21358927 +0000 UTC"
Jul 14 01:00:10.466666 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.466661    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14452h48m9.746931817s"
Jul 14 01:00:10.553662 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.553610    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.653959 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.653937    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.754301 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.754260    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.842024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:10.841803    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad49a4c43a161f6386858ac515eb52ef.slice/crio-44c205956b1294f6757bb0481a5a0e0eb3acd2e06cbfdd03544fda2c4797a9de WatchSource:0}: Error finding container 44c205956b1294f6757bb0481a5a0e0eb3acd2e06cbfdd03544fda2c4797a9de: Status 404 returned error can't find the container with id 44c205956b1294f6757bb0481a5a0e0eb3acd2e06cbfdd03544fda2c4797a9de
Jul 14 01:00:10.846830 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.846812    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 01:00:10.854548 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.854514    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:10.870744 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:10.870722    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 01:00:10.954896 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:10.954866    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-32.ec2.internal\" not found"
Jul 14 01:00:11.025915 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.025888    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 01:00:11.060034 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.060012    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:11.070885 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.070869    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 01:00:11.071893 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.071881    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal"
Jul 14 01:00:11.080747 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.080728    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 01:00:11.080827 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.080776    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3082df98acf7e090e04a68eb61aa6a.slice/crio-832103f9bb03f3f9be04b48c0482fb3b2c28ded1d55c1ad5c346386ca7b63e50 WatchSource:0}: Error finding container 832103f9bb03f3f9be04b48c0482fb3b2c28ded1d55c1ad5c346386ca7b63e50: Status 404 returned error can't find the container with id 832103f9bb03f3f9be04b48c0482fb3b2c28ded1d55c1ad5c346386ca7b63e50
Jul 14 01:00:11.255825 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.255808    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 01:00:11.442224 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.442161    2580 apiserver.go:52] "Watching apiserver"
Jul 14 01:00:11.447596 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.447575    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 01:00:11.447942 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.447922    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal","openshift-multus/multus-additional-cni-plugins-ksnzm","openshift-multus/multus-nx776","openshift-multus/network-metrics-daemon-x2jvq","openshift-network-diagnostics/network-check-target-kzbkf","openshift-network-operator/iptables-alerter-c2gps","kube-system/konnectivity-agent-h2hxt","kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal","openshift-cluster-node-tuning-operator/tuned-pd6jn","openshift-dns/node-resolver-ggkk8","openshift-image-registry/node-ca-lp74x","openshift-ovn-kubernetes/ovnkube-node-v5tc5","kube-system/global-pull-secret-syncer-lwmb9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"]
Jul 14 01:00:11.451017 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.450988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:11.452722 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.452699    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7gdfp\""
Jul 14 01:00:11.452806 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.452719    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 01:00:11.452806 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.452708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 01:00:11.453084 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.453065    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.453168 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.453119    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:11.455064 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.455047    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.456884 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.456861    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 01:00:11.457000 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.456982    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 01:00:11.457113 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.457003    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 01:00:11.457113 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.457054    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 01:00:11.457221 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.457202    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:11.457271 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.457256    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:11.457311 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.457269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9z4w8\""
Jul 14 01:00:11.459241 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.459222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:11.459309 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.459284    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:11.460551 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.460534    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 01:00:11.461417 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.461401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.462892 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.462876    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 01:00:11.463126 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.463098    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dzfzm\""
Jul 14 01:00:11.463190 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.463166    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 01:00:11.463190 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.463182    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 01:00:11.463473 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.463459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.465700 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.465560    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 01:00:11.465700 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.465590    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6gcnj\""
Jul 14 01:00:11.465700 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.465630    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 01:00:11.465886 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.465786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.467179 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.467163    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 01:00:11.467378 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.467365    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 01:00:11.467476 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.467462    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8nbln\""
Jul 14 01:00:11.467542 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.467513    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 01:00:11.468107 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.468092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.468891 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.468877    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 01:00:11.469449 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.469433    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 01:00:11.469533 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.469506    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f27tz\""
Jul 14 01:00:11.469597 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.469433    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 01:00:11.470418 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.470402    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.471078 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d6a1433-d511-4afe-9860-ca3ca555fe54-dbus\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.471144 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-etc-kubernetes\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471255 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc2sp\" (UniqueName: \"kubernetes.io/projected/628a13aa-d3a0-4ebc-843e-45aafd8338bc-kube-api-access-hc2sp\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471255 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:11.471255 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.471255 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-system-cni-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471402 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-netns\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471402 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-cni-bin\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471402 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-kubelet\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471402 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-cni-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471402 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-os-release\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:11.471578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l272w\" (UniqueName: \"kubernetes.io/projected/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-kube-api-access-l272w\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:11.471578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-multus-certs\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b619f9b-7574-4199-8d79-01fd455f9829-konnectivity-ca\") pod \"konnectivity-agent-h2hxt\" (UID: \"0b619f9b-7574-4199-8d79-01fd455f9829\") " pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:11.471578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27c93d4c-0917-4083-82a5-f07c4ab5241f-host-slash\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.471578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628a13aa-d3a0-4ebc-843e-45aafd8338bc-cni-binary-copy\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-cni-multus\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/27c93d4c-0917-4083-82a5-f07c4ab5241f-iptables-alerter-script\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b619f9b-7574-4199-8d79-01fd455f9829-agent-certs\") pod \"konnectivity-agent-h2hxt\" (UID: \"0b619f9b-7574-4199-8d79-01fd455f9829\") " pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d6a1433-d511-4afe-9860-ca3ca555fe54-kubelet-config\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-daemon-config\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-cnibin\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-socket-dir-parent\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-k8s-cni-cncf-io\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-hostroot\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-conf-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.471834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.471800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlbcl\" (UniqueName: \"kubernetes.io/projected/27c93d4c-0917-4083-82a5-f07c4ab5241f-kube-api-access-zlbcl\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.472093 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.472040    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mv7w7\""
Jul 14 01:00:11.472293 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.472278    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 01:00:11.472465 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.472446    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 01:00:11.472703 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.472689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.474122 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.474106    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 01:00:11.474288 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.474270    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-stz8p\""
Jul 14 01:00:11.474339 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.474287    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 01:00:11.474383 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.474289    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 01:00:11.475010 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.474993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.476908 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.476892    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 01:00:11.477354 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.477332    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 01:00:11.477508 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.477489    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 01:00:11.477832 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.477745    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 01:00:11.477832 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.477806    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tr6kk\""
Jul 14 01:00:11.478014 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.477996    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 01:00:11.479257 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.478261    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 01:00:11.491535 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.491504    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qnv7t"
Jul 14 01:00:11.498248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.498230    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qnv7t"
Jul 14 01:00:11.560870 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.560855    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 01:00:11.572930 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.572913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628a13aa-d3a0-4ebc-843e-45aafd8338bc-cni-binary-copy\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573037 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.572947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-cni-multus\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573037 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.572967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-kubernetes\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.573037 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-cni-multus\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573193 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-lib-modules\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.573193 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p475h\" (UniqueName: \"kubernetes.io/projected/a25163da-e9be-4dc1-9707-e86477831a4d-kube-api-access-p475h\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.573193 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.573193 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d6a1433-d511-4afe-9860-ca3ca555fe54-kubelet-config\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.573193 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cnibin\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d6a1433-d511-4afe-9860-ca3ca555fe54-kubelet-config\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-run-netns\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-cnibin\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-k8s-cni-cncf-io\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-host\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628a13aa-d3a0-4ebc-843e-45aafd8338bc-cni-binary-copy\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-device-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-cnibin\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-k8s-cni-cncf-io\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-log-socket\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-env-overrides\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wd7w\" (UniqueName: \"kubernetes.io/projected/6569e768-b512-4ed8-8332-69b769ce7f95-kube-api-access-9wd7w\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d6a1433-d511-4afe-9860-ca3ca555fe54-dbus\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-etc-kubernetes\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-systemd\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a25163da-e9be-4dc1-9707-e86477831a4d-tmp\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6qxc\" (UniqueName: \"kubernetes.io/projected/944c1130-84b2-43f0-9eea-df2ecf82b912-kube-api-access-h6qxc\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d6a1433-d511-4afe-9860-ca3ca555fe54-dbus\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm4kw\" (UniqueName: \"kubernetes.io/projected/e2073ac1-0532-4124-a3ef-d37024685bc1-kube-api-access-sm4kw\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-etc-kubernetes\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-system-cni-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-netns\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.573690    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-cni-bin\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.573929 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.573742    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret podName:6d6a1433-d511-4afe-9860-ca3ca555fe54 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:12.073724747 +0000 UTC m=+2.968575387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret") pod "global-pull-secret-syncer-lwmb9" (UID: "6d6a1433-d511-4afe-9860-ca3ca555fe54") : object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573744    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-system-cni-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-netns\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cni-binary-copy\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2073ac1-0532-4124-a3ef-d37024685bc1-hosts-file\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-cni-bin\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-cni-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l272w\" (UniqueName: \"kubernetes.io/projected/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-kube-api-access-l272w\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-modprobe-d\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2073ac1-0532-4124-a3ef-d37024685bc1-tmp-dir\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-cni-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20434812-dd9a-43d5-8eff-612bbeff893d-serviceca\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-systemd-units\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-etc-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.573939    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.573982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzd2r\" (UniqueName: \"kubernetes.io/projected/20434812-dd9a-43d5-8eff-612bbeff893d-kube-api-access-qzd2r\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.574732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-cni-netd\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.574037    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:00:12.074020369 +0000 UTC m=+2.968870990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/27c93d4c-0917-4083-82a5-f07c4ab5241f-iptables-alerter-script\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b619f9b-7574-4199-8d79-01fd455f9829-agent-certs\") pod \"konnectivity-agent-h2hxt\" (UID: \"0b619f9b-7574-4199-8d79-01fd455f9829\") " pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-socket-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrtjx\" (UniqueName: \"kubernetes.io/projected/ccc2deb6-ef0c-4606-9e80-2287b626ee13-kube-api-access-hrtjx\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-systemd\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-var-lib-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-cni-bin\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-daemon-config\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-run\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-var-lib-kubelet\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a25163da-e9be-4dc1-9707-e86477831a4d-etc-tuned\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-socket-dir-parent\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-hostroot\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574466    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-socket-dir-parent\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-conf-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-conf-dir\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysctl-d\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysctl-conf\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-hostroot\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-sys\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-os-release\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-node-log\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlbcl\" (UniqueName: \"kubernetes.io/projected/27c93d4c-0917-4083-82a5-f07c4ab5241f-kube-api-access-zlbcl\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-sys-fs\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc2sp\" (UniqueName: \"kubernetes.io/projected/628a13aa-d3a0-4ebc-843e-45aafd8338bc-kube-api-access-hc2sp\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574695    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/27c93d4c-0917-4083-82a5-f07c4ab5241f-iptables-alerter-script\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574710    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20434812-dd9a-43d5-8eff-612bbeff893d-host\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-kubelet\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.575873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-system-cni-dir\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574878    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/628a13aa-d3a0-4ebc-843e-45aafd8338bc-multus-daemon-config\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-var-lib-kubelet\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-slash\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-ovn\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6569e768-b512-4ed8-8332-69b769ce7f95-ovn-node-metrics-cert\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-ovnkube-script-lib\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.574983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-os-release\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-registration-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-kubelet\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-os-release\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-run-ovn-kubernetes\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-ovnkube-config\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-multus-certs\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b619f9b-7574-4199-8d79-01fd455f9829-konnectivity-ca\") pod \"konnectivity-agent-h2hxt\" (UID: \"0b619f9b-7574-4199-8d79-01fd455f9829\") " pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysconfig\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.576359 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.576910 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.576910 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/628a13aa-d3a0-4ebc-843e-45aafd8338bc-host-run-multus-certs\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.576910 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27c93d4c-0917-4083-82a5-f07c4ab5241f-host-slash\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.576910 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27c93d4c-0917-4083-82a5-f07c4ab5241f-host-slash\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.576910 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.575823    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b619f9b-7574-4199-8d79-01fd455f9829-konnectivity-ca\") pod \"konnectivity-agent-h2hxt\" (UID: \"0b619f9b-7574-4199-8d79-01fd455f9829\") " pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:11.577931 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.577916    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b619f9b-7574-4199-8d79-01fd455f9829-agent-certs\") pod \"konnectivity-agent-h2hxt\" (UID: \"0b619f9b-7574-4199-8d79-01fd455f9829\") " pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:11.581808 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.581794    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 01:00:11.581861 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.581813    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 01:00:11.581861 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.581822    2580 projected.go:196] Error preparing data for projected volume kube-api-access-9gnb6 for pod openshift-network-diagnostics/network-check-target-kzbkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:11.581920 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:11.581866    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6 podName:73ae53aa-354f-40c4-8c08-855b02fe3b4c nodeName:}" failed. No retries permitted until 2026-07-14 01:00:12.081855601 +0000 UTC m=+2.976706248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9gnb6" (UniqueName: "kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6") pod "network-check-target-kzbkf" (UID: "73ae53aa-354f-40c4-8c08-855b02fe3b4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:11.584152 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.584126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l272w\" (UniqueName: \"kubernetes.io/projected/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-kube-api-access-l272w\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:11.585016 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.585000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlbcl\" (UniqueName: \"kubernetes.io/projected/27c93d4c-0917-4083-82a5-f07c4ab5241f-kube-api-access-zlbcl\") pod \"iptables-alerter-c2gps\" (UID: \"27c93d4c-0917-4083-82a5-f07c4ab5241f\") " pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.585557 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.585540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc2sp\" (UniqueName: \"kubernetes.io/projected/628a13aa-d3a0-4ebc-843e-45aafd8338bc-kube-api-access-hc2sp\") pod \"multus-nx776\" (UID: \"628a13aa-d3a0-4ebc-843e-45aafd8338bc\") " pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.645373 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.645326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal" event={"ID":"9d3082df98acf7e090e04a68eb61aa6a","Type":"ContainerStarted","Data":"832103f9bb03f3f9be04b48c0482fb3b2c28ded1d55c1ad5c346386ca7b63e50"}
Jul 14 01:00:11.646417 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.646392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal" event={"ID":"ad49a4c43a161f6386858ac515eb52ef","Type":"ContainerStarted","Data":"44c205956b1294f6757bb0481a5a0e0eb3acd2e06cbfdd03544fda2c4797a9de"}
Jul 14 01:00:11.676045 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-registration-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.676150 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-kubelet\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676150 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-run-ovn-kubernetes\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676150 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676150 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-ovnkube-config\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-kubelet\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-registration-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-run-ovn-kubernetes\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysconfig\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysconfig\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-kubernetes\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-lib-modules\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p475h\" (UniqueName: \"kubernetes.io/projected/a25163da-e9be-4dc1-9707-e86477831a4d-kube-api-access-p475h\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-kubernetes\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cnibin\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.676448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-run-netns\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-host\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-device-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-lib-modules\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-log-socket\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-env-overrides\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-host\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wd7w\" (UniqueName: \"kubernetes.io/projected/6569e768-b512-4ed8-8332-69b769ce7f95-kube-api-access-9wd7w\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-device-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-run-netns\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-systemd\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a25163da-e9be-4dc1-9707-e86477831a4d-tmp\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6qxc\" (UniqueName: \"kubernetes.io/projected/944c1130-84b2-43f0-9eea-df2ecf82b912-kube-api-access-h6qxc\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm4kw\" (UniqueName: \"kubernetes.io/projected/e2073ac1-0532-4124-a3ef-d37024685bc1-kube-api-access-sm4kw\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cni-binary-copy\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.677329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2073ac1-0532-4124-a3ef-d37024685bc1-hosts-file\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-modprobe-d\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2073ac1-0532-4124-a3ef-d37024685bc1-tmp-dir\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-env-overrides\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20434812-dd9a-43d5-8eff-612bbeff893d-serviceca\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-systemd-units\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.676997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-etc-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzd2r\" (UniqueName: \"kubernetes.io/projected/20434812-dd9a-43d5-8eff-612bbeff893d-kube-api-access-qzd2r\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-log-socket\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-cni-netd\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677102    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2073ac1-0532-4124-a3ef-d37024685bc1-hosts-file\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-systemd-units\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cnibin\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-etc-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-cni-netd\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-socket-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.678106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrtjx\" (UniqueName: \"kubernetes.io/projected/ccc2deb6-ef0c-4606-9e80-2287b626ee13-kube-api-access-hrtjx\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2073ac1-0532-4124-a3ef-d37024685bc1-tmp-dir\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-modprobe-d\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-ovnkube-config\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-systemd\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-systemd\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-var-lib-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-systemd\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-socket-dir\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20434812-dd9a-43d5-8eff-612bbeff893d-serviceca\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-cni-bin\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-var-lib-openvswitch\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-run\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677594    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-cni-bin\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cni-binary-copy\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-var-lib-kubelet\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-var-lib-kubelet\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.678909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-run\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a25163da-e9be-4dc1-9707-e86477831a4d-etc-tuned\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysctl-d\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysctl-conf\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-sys\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-os-release\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysctl-d\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-node-log\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-node-log\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677863    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-sys\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-sys-fs\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a25163da-e9be-4dc1-9707-e86477831a4d-etc-sysctl-conf\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-os-release\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20434812-dd9a-43d5-8eff-612bbeff893d-host\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/944c1130-84b2-43f0-9eea-df2ecf82b912-sys-fs\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.679828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-system-cni-dir\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.677994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20434812-dd9a-43d5-8eff-612bbeff893d-host\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-slash\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-ovn\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccc2deb6-ef0c-4606-9e80-2287b626ee13-system-cni-dir\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6569e768-b512-4ed8-8332-69b769ce7f95-ovn-node-metrics-cert\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-ovnkube-script-lib\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678100    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-run-ovn\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6569e768-b512-4ed8-8332-69b769ce7f95-host-slash\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ccc2deb6-ef0c-4606-9e80-2287b626ee13-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.678514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6569e768-b512-4ed8-8332-69b769ce7f95-ovnkube-script-lib\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.679917    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a25163da-e9be-4dc1-9707-e86477831a4d-tmp\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.680267    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6569e768-b512-4ed8-8332-69b769ce7f95-ovn-node-metrics-cert\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.680716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.680362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a25163da-e9be-4dc1-9707-e86477831a4d-etc-tuned\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.684804 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.684781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzd2r\" (UniqueName: \"kubernetes.io/projected/20434812-dd9a-43d5-8eff-612bbeff893d-kube-api-access-qzd2r\") pod \"node-ca-lp74x\" (UID: \"20434812-dd9a-43d5-8eff-612bbeff893d\") " pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.684893 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.684772    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p475h\" (UniqueName: \"kubernetes.io/projected/a25163da-e9be-4dc1-9707-e86477831a4d-kube-api-access-p475h\") pod \"tuned-pd6jn\" (UID: \"a25163da-e9be-4dc1-9707-e86477831a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.686485 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.686460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrtjx\" (UniqueName: \"kubernetes.io/projected/ccc2deb6-ef0c-4606-9e80-2287b626ee13-kube-api-access-hrtjx\") pod \"multus-additional-cni-plugins-ksnzm\" (UID: \"ccc2deb6-ef0c-4606-9e80-2287b626ee13\") " pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.686851 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.686828    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm4kw\" (UniqueName: \"kubernetes.io/projected/e2073ac1-0532-4124-a3ef-d37024685bc1-kube-api-access-sm4kw\") pod \"node-resolver-ggkk8\" (UID: \"e2073ac1-0532-4124-a3ef-d37024685bc1\") " pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.686851 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.686843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6qxc\" (UniqueName: \"kubernetes.io/projected/944c1130-84b2-43f0-9eea-df2ecf82b912-kube-api-access-h6qxc\") pod \"aws-ebs-csi-driver-node-fdxkm\" (UID: \"944c1130-84b2-43f0-9eea-df2ecf82b912\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.688577 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.688536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wd7w\" (UniqueName: \"kubernetes.io/projected/6569e768-b512-4ed8-8332-69b769ce7f95-kube-api-access-9wd7w\") pod \"ovnkube-node-v5tc5\" (UID: \"6569e768-b512-4ed8-8332-69b769ce7f95\") " pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.760497 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.760435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:11.768040 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.768012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nx776"
Jul 14 01:00:11.772270 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.772248    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b619f9b_7574_4199_8d79_01fd455f9829.slice/crio-5363c4b50c8c0019578adecde368d094c23c202e9bc4e0e5419d27327b2395a9 WatchSource:0}: Error finding container 5363c4b50c8c0019578adecde368d094c23c202e9bc4e0e5419d27327b2395a9: Status 404 returned error can't find the container with id 5363c4b50c8c0019578adecde368d094c23c202e9bc4e0e5419d27327b2395a9
Jul 14 01:00:11.776850 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.776818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c2gps"
Jul 14 01:00:11.778449 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.778424    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628a13aa_d3a0_4ebc_843e_45aafd8338bc.slice/crio-462e4de5189da31ecb9f0ec61579d147d6ba03c1139eb9301c07b5f6e4bf83e6 WatchSource:0}: Error finding container 462e4de5189da31ecb9f0ec61579d147d6ba03c1139eb9301c07b5f6e4bf83e6: Status 404 returned error can't find the container with id 462e4de5189da31ecb9f0ec61579d147d6ba03c1139eb9301c07b5f6e4bf83e6
Jul 14 01:00:11.783556 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.783536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pd6jn"
Jul 14 01:00:11.787281 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.787250    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c93d4c_0917_4083_82a5_f07c4ab5241f.slice/crio-4ed6f1541319e1dea6e5625845f669863d4c51a66700c172c25ba6801b6a2c98 WatchSource:0}: Error finding container 4ed6f1541319e1dea6e5625845f669863d4c51a66700c172c25ba6801b6a2c98: Status 404 returned error can't find the container with id 4ed6f1541319e1dea6e5625845f669863d4c51a66700c172c25ba6801b6a2c98
Jul 14 01:00:11.788197 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.788179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm"
Jul 14 01:00:11.793510 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.793469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ksnzm"
Jul 14 01:00:11.793800 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.793779    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25163da_e9be_4dc1_9707_e86477831a4d.slice/crio-2c6a23fd7ce1bf8dcb72e42ed8482d9483866fa169dad82ffa3487a7d42dcd67 WatchSource:0}: Error finding container 2c6a23fd7ce1bf8dcb72e42ed8482d9483866fa169dad82ffa3487a7d42dcd67: Status 404 returned error can't find the container with id 2c6a23fd7ce1bf8dcb72e42ed8482d9483866fa169dad82ffa3487a7d42dcd67
Jul 14 01:00:11.799472 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.799447    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ggkk8"
Jul 14 01:00:11.801862 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.801841    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944c1130_84b2_43f0_9eea_df2ecf82b912.slice/crio-966a202d06f0a70859d7196bebbdc91e2039219381d146fe6bc4eca9f1e99f18 WatchSource:0}: Error finding container 966a202d06f0a70859d7196bebbdc91e2039219381d146fe6bc4eca9f1e99f18: Status 404 returned error can't find the container with id 966a202d06f0a70859d7196bebbdc91e2039219381d146fe6bc4eca9f1e99f18
Jul 14 01:00:11.804579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.804498    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lp74x"
Jul 14 01:00:11.806902 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.806888    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc2deb6_ef0c_4606_9e80_2287b626ee13.slice/crio-ecf9743073bea9cbb70e95dfece35b59545dd6157117477231cb9804bc915907 WatchSource:0}: Error finding container ecf9743073bea9cbb70e95dfece35b59545dd6157117477231cb9804bc915907: Status 404 returned error can't find the container with id ecf9743073bea9cbb70e95dfece35b59545dd6157117477231cb9804bc915907
Jul 14 01:00:11.809571 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:11.809015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:11.810024 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.810000    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2073ac1_0532_4124_a3ef_d37024685bc1.slice/crio-1c50853dd53f04d9e11e2db78a7b4d61e266ade4d23bc1c475a5c9f10627cc86 WatchSource:0}: Error finding container 1c50853dd53f04d9e11e2db78a7b4d61e266ade4d23bc1c475a5c9f10627cc86: Status 404 returned error can't find the container with id 1c50853dd53f04d9e11e2db78a7b4d61e266ade4d23bc1c475a5c9f10627cc86
Jul 14 01:00:11.820185 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.820165    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6569e768_b512_4ed8_8332_69b769ce7f95.slice/crio-18fa0f79a78342f58d8d2cdb7a6ae7c54272d373d965fa4b5997e0595860f10c WatchSource:0}: Error finding container 18fa0f79a78342f58d8d2cdb7a6ae7c54272d373d965fa4b5997e0595860f10c: Status 404 returned error can't find the container with id 18fa0f79a78342f58d8d2cdb7a6ae7c54272d373d965fa4b5997e0595860f10c
Jul 14 01:00:11.822202 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:11.822181    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20434812_dd9a_43d5_8eff_612bbeff893d.slice/crio-1ce5b5da333302714e476180efb5c5ca1ab3608718e496e500300f48f310592c WatchSource:0}: Error finding container 1ce5b5da333302714e476180efb5c5ca1ab3608718e496e500300f48f310592c: Status 404 returned error can't find the container with id 1ce5b5da333302714e476180efb5c5ca1ab3608718e496e500300f48f310592c
Jul 14 01:00:12.080021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.079956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:12.080021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.079992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:12.080192 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.080120    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:12.080192 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.080157    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:12.080192 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.080170    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret podName:6d6a1433-d511-4afe-9860-ca3ca555fe54 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:13.080156328 +0000 UTC m=+3.975006945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret") pod "global-pull-secret-syncer-lwmb9" (UID: "6d6a1433-d511-4afe-9860-ca3ca555fe54") : object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:12.080320 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.080195    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:00:13.080184093 +0000 UTC m=+3.975034709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:12.181146 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.181110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:12.181308 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.181288    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 01:00:12.181393 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.181321    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 01:00:12.181393 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.181334    2580 projected.go:196] Error preparing data for projected volume kube-api-access-9gnb6 for pod openshift-network-diagnostics/network-check-target-kzbkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:12.181498 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.181393    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6 podName:73ae53aa-354f-40c4-8c08-855b02fe3b4c nodeName:}" failed. No retries permitted until 2026-07-14 01:00:13.18137419 +0000 UTC m=+4.076224825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gnb6" (UniqueName: "kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6") pod "network-check-target-kzbkf" (UID: "73ae53aa-354f-40c4-8c08-855b02fe3b4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:12.501674 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.501586    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 00:55:11 +0000 UTC" deadline="2028-01-29 20:06:18.371340258 +0000 UTC"
Jul 14 01:00:12.501674 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.501622    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13555h6m5.869723098s"
Jul 14 01:00:12.636845 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.636179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:12.636845 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.636297    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:12.636845 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.636697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:12.636845 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:12.636794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:12.649472 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.649441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" event={"ID":"ccc2deb6-ef0c-4606-9e80-2287b626ee13","Type":"ContainerStarted","Data":"ecf9743073bea9cbb70e95dfece35b59545dd6157117477231cb9804bc915907"}
Jul 14 01:00:12.650975 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.650859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm" event={"ID":"944c1130-84b2-43f0-9eea-df2ecf82b912","Type":"ContainerStarted","Data":"966a202d06f0a70859d7196bebbdc91e2039219381d146fe6bc4eca9f1e99f18"}
Jul 14 01:00:12.652361 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.652296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c2gps" event={"ID":"27c93d4c-0917-4083-82a5-f07c4ab5241f","Type":"ContainerStarted","Data":"4ed6f1541319e1dea6e5625845f669863d4c51a66700c172c25ba6801b6a2c98"}
Jul 14 01:00:12.654305 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.654242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h2hxt" event={"ID":"0b619f9b-7574-4199-8d79-01fd455f9829","Type":"ContainerStarted","Data":"5363c4b50c8c0019578adecde368d094c23c202e9bc4e0e5419d27327b2395a9"}
Jul 14 01:00:12.656176 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.656153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lp74x" event={"ID":"20434812-dd9a-43d5-8eff-612bbeff893d","Type":"ContainerStarted","Data":"1ce5b5da333302714e476180efb5c5ca1ab3608718e496e500300f48f310592c"}
Jul 14 01:00:12.657629 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.657542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"18fa0f79a78342f58d8d2cdb7a6ae7c54272d373d965fa4b5997e0595860f10c"}
Jul 14 01:00:12.659136 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.659068    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pd6jn" event={"ID":"a25163da-e9be-4dc1-9707-e86477831a4d","Type":"ContainerStarted","Data":"2c6a23fd7ce1bf8dcb72e42ed8482d9483866fa169dad82ffa3487a7d42dcd67"}
Jul 14 01:00:12.660569 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.660481    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nx776" event={"ID":"628a13aa-d3a0-4ebc-843e-45aafd8338bc","Type":"ContainerStarted","Data":"462e4de5189da31ecb9f0ec61579d147d6ba03c1139eb9301c07b5f6e4bf83e6"}
Jul 14 01:00:12.662029 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:12.661971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ggkk8" event={"ID":"e2073ac1-0532-4124-a3ef-d37024685bc1","Type":"ContainerStarted","Data":"1c50853dd53f04d9e11e2db78a7b4d61e266ade4d23bc1c475a5c9f10627cc86"}
Jul 14 01:00:13.088622 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:13.088451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:13.088622 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:13.088508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:13.088853 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.088702    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:13.088853 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.088759    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:00:15.088740493 +0000 UTC m=+5.983591113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:13.089246 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.089158    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:13.089246 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.089207    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret podName:6d6a1433-d511-4afe-9860-ca3ca555fe54 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:15.089192495 +0000 UTC m=+5.984043115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret") pod "global-pull-secret-syncer-lwmb9" (UID: "6d6a1433-d511-4afe-9860-ca3ca555fe54") : object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:13.189073 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:13.189031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:13.189281 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.189264    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 01:00:13.189377 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.189291    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 01:00:13.189377 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.189305    2580 projected.go:196] Error preparing data for projected volume kube-api-access-9gnb6 for pod openshift-network-diagnostics/network-check-target-kzbkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:13.189377 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.189361    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6 podName:73ae53aa-354f-40c4-8c08-855b02fe3b4c nodeName:}" failed. No retries permitted until 2026-07-14 01:00:15.189342679 +0000 UTC m=+6.084193314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gnb6" (UniqueName: "kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6") pod "network-check-target-kzbkf" (UID: "73ae53aa-354f-40c4-8c08-855b02fe3b4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:13.501887 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:13.501797    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 00:55:11 +0000 UTC" deadline="2028-03-16 18:22:45.20793038 +0000 UTC"
Jul 14 01:00:13.501887 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:13.501842    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14681h22m31.706092413s"
Jul 14 01:00:13.641561 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:13.636483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:13.641561 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:13.636770    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:14.637247 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:14.637208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:14.637725 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:14.637330    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:14.637787 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:14.637730    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:14.637894 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:14.637823    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:15.107066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:15.106973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:15.107066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:15.107028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:15.107290 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.107168    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:15.107290 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.107226    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:00:19.107206706 +0000 UTC m=+10.002057338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:15.107383 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.107293    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:15.107383 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.107327    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret podName:6d6a1433-d511-4afe-9860-ca3ca555fe54 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:19.107315878 +0000 UTC m=+10.002166498 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret") pod "global-pull-secret-syncer-lwmb9" (UID: "6d6a1433-d511-4afe-9860-ca3ca555fe54") : object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:15.208469 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:15.207803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:15.208469 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.208022    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 01:00:15.208469 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.208045    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 01:00:15.208469 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.208057    2580 projected.go:196] Error preparing data for projected volume kube-api-access-9gnb6 for pod openshift-network-diagnostics/network-check-target-kzbkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:15.208469 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.208111    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6 podName:73ae53aa-354f-40c4-8c08-855b02fe3b4c nodeName:}" failed. No retries permitted until 2026-07-14 01:00:19.208094187 +0000 UTC m=+10.102944810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gnb6" (UniqueName: "kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6") pod "network-check-target-kzbkf" (UID: "73ae53aa-354f-40c4-8c08-855b02fe3b4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:15.639153 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:15.639122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:15.639560 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:15.639264    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:16.637213 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:16.637179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:16.637368 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:16.637291    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:16.637613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:16.637594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:16.637722 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:16.637700    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:17.637775 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:17.637687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:17.638189 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:17.637834    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:18.636333 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:18.636301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:18.636502 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:18.636428    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:18.636502 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:18.636465    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:18.636648 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:18.636579    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:19.140333 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:19.140064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:19.140333 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:19.140114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:19.140333 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.140240    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:19.140333 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.140244    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:19.140333 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.140301    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:00:27.140287732 +0000 UTC m=+18.035138362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:19.140333 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.140315    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret podName:6d6a1433-d511-4afe-9860-ca3ca555fe54 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:27.140308585 +0000 UTC m=+18.035159202 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret") pod "global-pull-secret-syncer-lwmb9" (UID: "6d6a1433-d511-4afe-9860-ca3ca555fe54") : object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:19.241445 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:19.241409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:19.241638 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.241587    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 01:00:19.241638 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.241607    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 01:00:19.241638 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.241618    2580 projected.go:196] Error preparing data for projected volume kube-api-access-9gnb6 for pod openshift-network-diagnostics/network-check-target-kzbkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:19.241794 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.241674    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6 podName:73ae53aa-354f-40c4-8c08-855b02fe3b4c nodeName:}" failed. No retries permitted until 2026-07-14 01:00:27.241656702 +0000 UTC m=+18.136507343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gnb6" (UniqueName: "kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6") pod "network-check-target-kzbkf" (UID: "73ae53aa-354f-40c4-8c08-855b02fe3b4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:19.637623 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:19.637513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:19.637775 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:19.637663    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:20.636791 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:20.636754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:20.637278 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:20.636880    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:20.637278 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:20.637264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:20.637392 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:20.637353    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:21.639048 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:21.638376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:21.639048 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:21.638493    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:22.636815 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:22.636786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:22.636993 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:22.636786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:22.636993 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:22.636890    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:22.637096 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:22.637003    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:23.636954 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:23.636864    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:23.637396 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:23.637007    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:24.636707 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:24.636671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:24.636889 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:24.636718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:24.636889 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:24.636794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:24.637002 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:24.636907    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:25.636805 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:25.636773    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:25.636955 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:25.636907    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:26.637030 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:26.636996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:26.637432 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:26.636999    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:26.637432 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:26.637133    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:26.637432 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:26.637190    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:27.199934 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:27.199907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:27.200180 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:27.199941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:27.200180 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.200049    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:27.200180 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.200055    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:27.200180 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.200117    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:00:43.200097308 +0000 UTC m=+34.094947925 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:27.200180 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.200140    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret podName:6d6a1433-d511-4afe-9860-ca3ca555fe54 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:43.200127498 +0000 UTC m=+34.094978115 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret") pod "global-pull-secret-syncer-lwmb9" (UID: "6d6a1433-d511-4afe-9860-ca3ca555fe54") : object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:27.300946 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:27.300916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:27.301116 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.301054    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 01:00:27.301116 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.301069    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 01:00:27.301116 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.301077    2580 projected.go:196] Error preparing data for projected volume kube-api-access-9gnb6 for pod openshift-network-diagnostics/network-check-target-kzbkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:27.301234 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.301153    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6 podName:73ae53aa-354f-40c4-8c08-855b02fe3b4c nodeName:}" failed. No retries permitted until 2026-07-14 01:00:43.30113701 +0000 UTC m=+34.195987642 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gnb6" (UniqueName: "kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6") pod "network-check-target-kzbkf" (UID: "73ae53aa-354f-40c4-8c08-855b02fe3b4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:27.637009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:27.636932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:27.637173 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:27.637084    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:28.636915 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:28.636880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:28.637112 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:28.637017    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:28.637411 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:28.637387    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:28.637812 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:28.637479    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:29.637493 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:29.637465    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:29.637906 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:29.637581    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:30.637124 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:30.637094    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:30.637301 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:30.637096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:30.637301 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:30.637214    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:30.637390 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:30.637302    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:31.636470 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:31.636441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:31.636852 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:31.636592    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:32.637263 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.636976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:32.637827 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.637034    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:32.637827 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:32.637312    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:32.637827 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:32.637432    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:32.702164 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.702137    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccc2deb6-ef0c-4606-9e80-2287b626ee13" containerID="cb6aadb1bd67d2c7ffa491e30450d36eeec8657ec8e3eb7ad8ee1c62284870be" exitCode=0
Jul 14 01:00:32.702237 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.702204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" event={"ID":"ccc2deb6-ef0c-4606-9e80-2287b626ee13","Type":"ContainerDied","Data":"cb6aadb1bd67d2c7ffa491e30450d36eeec8657ec8e3eb7ad8ee1c62284870be"}
Jul 14 01:00:32.705892 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.705866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm" event={"ID":"944c1130-84b2-43f0-9eea-df2ecf82b912","Type":"ContainerStarted","Data":"ae5368715016939df4394e6473c72ca2a7f53e57c9c70faf2639d377b2d35e03"}
Jul 14 01:00:32.708215 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.708191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h2hxt" event={"ID":"0b619f9b-7574-4199-8d79-01fd455f9829","Type":"ContainerStarted","Data":"103bcbee301847d499b082921c46adf6708f45638f6a58a1f069df4ff2f10001"}
Jul 14 01:00:32.709603 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.709582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal" event={"ID":"9d3082df98acf7e090e04a68eb61aa6a","Type":"ContainerStarted","Data":"ecd7ed2192376f7862b4040b2caf9808a9b5372a80a05cc19c8e434840c4ec48"}
Jul 14 01:00:32.710988 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.710964    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad49a4c43a161f6386858ac515eb52ef" containerID="e2092b8e30c5f80df502e3045d0755034ad077b9b80279fec67cf61d3380e6de" exitCode=0
Jul 14 01:00:32.711083 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.711003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal" event={"ID":"ad49a4c43a161f6386858ac515eb52ef","Type":"ContainerDied","Data":"e2092b8e30c5f80df502e3045d0755034ad077b9b80279fec67cf61d3380e6de"}
Jul 14 01:00:32.712366 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.712344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lp74x" event={"ID":"20434812-dd9a-43d5-8eff-612bbeff893d","Type":"ContainerStarted","Data":"e0915079fe7948251311b427df12cd4968ad894fa52bc4c5123019697ee55558"}
Jul 14 01:00:32.715150 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.715128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"4b8f44e1599e569356323838002e5b770d272863c6c99543a7c4d05226aff8ae"}
Jul 14 01:00:32.715245 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.715199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"caf7d181f71198a58efbb39221a9a76946489b50240270d520dc7bad663d8a6b"}
Jul 14 01:00:32.715301 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.715251    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"dd2e1b7b7cf4ad18d7f2eb26d8573987e21e49806ff299a4f47ba62b55a46ccc"}
Jul 14 01:00:32.715356 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.715300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"3f1e2c6187e1b7bcf40f4a9a768f5ae330f9c652fca6ff0c4e16c846b3597343"}
Jul 14 01:00:32.715356 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.715348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"45b22a7232dd8e2a1ef9e2640681d4d4bbe91c7bca5ba047b6e8d0e5c658b6f1"}
Jul 14 01:00:32.715452 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.715394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"00e7846ac07b197ee4e3dd77f1b0afea182a6671ac1551d8a0240ae3dde8a7b5"}
Jul 14 01:00:32.716401 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.716383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pd6jn" event={"ID":"a25163da-e9be-4dc1-9707-e86477831a4d","Type":"ContainerStarted","Data":"1f464641d005a2ea0ce1c5d26858fbb66e93fb60838c9469b3e4f60a9125dfc2"}
Jul 14 01:00:32.717909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.717889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nx776" event={"ID":"628a13aa-d3a0-4ebc-843e-45aafd8338bc","Type":"ContainerStarted","Data":"60cb5355a79eb7e186f6adea81e285d5f379edd62f2b14770a208bd4caeb10f2"}
Jul 14 01:00:32.719033 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.719010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ggkk8" event={"ID":"e2073ac1-0532-4124-a3ef-d37024685bc1","Type":"ContainerStarted","Data":"7282931c9fcfb5555f4487bc0b731a53e98cff21e6df28a02dd9d9b5dec133bd"}
Jul 14 01:00:32.746162 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.745635    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ggkk8" podStartSLOduration=3.754166879 podStartE2EDuration="23.745621169s" totalImagesPullingTime="19.99145429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:32.733712834 +0000 UTC m=+23.628563474" watchObservedRunningTime="2026-07-14 01:00:32.745621169 +0000 UTC m=+23.640471805"
Jul 14 01:00:32.746162 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.745728    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lp74x" podStartSLOduration=12.351723941 podStartE2EDuration="23.745721989s" totalImagesPullingTime="11.393998048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:32.745324739 +0000 UTC m=+23.640175378" watchObservedRunningTime="2026-07-14 01:00:32.745721989 +0000 UTC m=+23.640572628"
Jul 14 01:00:32.789966 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.789926    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-32.ec2.internal" podStartSLOduration=21.789912705 podStartE2EDuration="21.789912705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:32.772215566 +0000 UTC m=+23.667066218" watchObservedRunningTime="2026-07-14 01:00:32.789912705 +0000 UTC m=+23.684763344"
Jul 14 01:00:32.803342 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.803309    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pd6jn" podStartSLOduration=3.790604017 podStartE2EDuration="23.803297488s" totalImagesPullingTime="20.012693471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:32.803196981 +0000 UTC m=+23.698047816" watchObservedRunningTime="2026-07-14 01:00:32.803297488 +0000 UTC m=+23.698148127"
Jul 14 01:00:32.803699 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:32.803677    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nx776" podStartSLOduration=3.764531419 podStartE2EDuration="23.803670852s" totalImagesPullingTime="20.039139433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:32.789764759 +0000 UTC m=+23.684615400" watchObservedRunningTime="2026-07-14 01:00:32.803670852 +0000 UTC m=+23.698521491"
Jul 14 01:00:33.636444 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:33.636372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:33.636646 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:33.636513    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:33.724464 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:33.724252    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal" event={"ID":"ad49a4c43a161f6386858ac515eb52ef","Type":"ContainerStarted","Data":"0cb6840e5d0cab15d1d6149dc73ce7b64228b7322595679109fc4b9364525943"}
Jul 14 01:00:33.726949 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:33.726918    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c2gps" event={"ID":"27c93d4c-0917-4083-82a5-f07c4ab5241f","Type":"ContainerStarted","Data":"016bfac59ca6ef9fa8ed8ea60ddea21dd9e1425e68086ee43feb485b0243ebfc"}
Jul 14 01:00:33.737874 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:33.737818    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h2hxt" podStartSLOduration=4.706307863 podStartE2EDuration="24.737800695s" totalImagesPullingTime="20.031492832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:32.816966207 +0000 UTC m=+23.711816846" watchObservedRunningTime="2026-07-14 01:00:33.737800695 +0000 UTC m=+24.632651337"
Jul 14 01:00:33.738660 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:33.738624    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-32.ec2.internal" podStartSLOduration=22.738611852 podStartE2EDuration="22.738611852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:33.737614538 +0000 UTC m=+24.632465181" watchObservedRunningTime="2026-07-14 01:00:33.738611852 +0000 UTC m=+24.633462492"
Jul 14 01:00:33.750505 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:33.750460    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c2gps" podStartSLOduration=4.73406295 podStartE2EDuration="24.750448596s" totalImagesPullingTime="20.016385646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:33.75015957 +0000 UTC m=+24.645010209" watchObservedRunningTime="2026-07-14 01:00:33.750448596 +0000 UTC m=+24.645299235"
Jul 14 01:00:33.809606 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:33.809559    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 01:00:34.532358 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:34.532248    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T01:00:33.809600846Z","UUID":"2ce0ff22-10be-4758-a754-1debb2df891a","Handler":null,"Name":"","Endpoint":""}
Jul 14 01:00:34.533956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:34.533936    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 01:00:34.534087 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:34.533977    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 01:00:34.636592 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:34.636564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:34.636735 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:34.636568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:34.636792 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:34.636734    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:34.636792 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:34.636655    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:34.732901 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:34.732868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm" event={"ID":"944c1130-84b2-43f0-9eea-df2ecf82b912","Type":"ContainerStarted","Data":"f0f4d625e914e4c36d5bde32669e5b5d8252119b34a192904a8f28383c5e6b9e"}
Jul 14 01:00:35.494600 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:35.494571    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:35.636624 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:35.636596    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:35.636763 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:35.636739    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:35.738711 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:35.738678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"dc0e88f5106177add4ec852a064a4689e385efbc4c385877508c553fa207f61b"}
Jul 14 01:00:35.740914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:35.740884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm" event={"ID":"944c1130-84b2-43f0-9eea-df2ecf82b912","Type":"ContainerStarted","Data":"31024d6d1e446b6180633074eb7ebb8c78b012d2a354d8715cc76d2d1dbe1e9b"}
Jul 14 01:00:35.758674 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:35.758602    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdxkm" podStartSLOduration=3.762968447 podStartE2EDuration="26.758587545s" totalImagesPullingTime="22.995619098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:35.758083607 +0000 UTC m=+26.652934245" watchObservedRunningTime="2026-07-14 01:00:35.758587545 +0000 UTC m=+26.653438184"
Jul 14 01:00:36.461101 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:36.461068    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:36.461939 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:36.461917    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:36.637078 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:36.637050    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:36.637224 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:36.637149    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:36.637224 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:36.637211    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:36.637335 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:36.637297    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:36.743416 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:36.743339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h2hxt"
Jul 14 01:00:37.637102 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:37.636933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:37.637210 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:37.637190    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:37.748276 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:37.748217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" event={"ID":"6569e768-b512-4ed8-8332-69b769ce7f95","Type":"ContainerStarted","Data":"71e69546047d8237f1aad5dc9232c554c262d6b3085eb2e9672d95cc16ca7cc3"}
Jul 14 01:00:37.748737 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:37.748652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:37.748797 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:37.748744    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:37.750251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:37.750228    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccc2deb6-ef0c-4606-9e80-2287b626ee13" containerID="140156750afdc1e62f8f444c49b6d37d8dbeefbf7f9d96c7fbaf7fa0ce8b0478" exitCode=0
Jul 14 01:00:37.750370 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:37.750311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" event={"ID":"ccc2deb6-ef0c-4606-9e80-2287b626ee13","Type":"ContainerDied","Data":"140156750afdc1e62f8f444c49b6d37d8dbeefbf7f9d96c7fbaf7fa0ce8b0478"}
Jul 14 01:00:37.764183 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:37.764166    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:37.770910 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:37.770871    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5" podStartSLOduration=8.601364125 podStartE2EDuration="28.770858334s" totalImagesPullingTime="20.169494209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:37.770541175 +0000 UTC m=+28.665391815" watchObservedRunningTime="2026-07-14 01:00:37.770858334 +0000 UTC m=+28.665708970"
Jul 14 01:00:38.637196 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:38.637152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:38.637420 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:38.637295    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:38.637793 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:38.637758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:38.638060 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:38.638038    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:38.753071 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:38.753036    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:38.768176 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:38.768153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:00:39.349553 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:39.349239    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lwmb9"]
Jul 14 01:00:39.349553 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:39.349441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kzbkf"]
Jul 14 01:00:39.349553 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:39.349470    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x2jvq"]
Jul 14 01:00:39.349788 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:39.349588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:39.349788 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:39.349604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:39.349788 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:39.349711    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:39.349926 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:39.349851    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:39.349977 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:39.349943    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:39.350033 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:39.350000    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:39.756487 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:39.756410    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccc2deb6-ef0c-4606-9e80-2287b626ee13" containerID="4be8b1bda957a0b8594bbdb93fcfb36767b6ee6bb5f6bca5214f13f5aede4fc8" exitCode=0
Jul 14 01:00:39.756968 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:39.756494    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" event={"ID":"ccc2deb6-ef0c-4606-9e80-2287b626ee13","Type":"ContainerDied","Data":"4be8b1bda957a0b8594bbdb93fcfb36767b6ee6bb5f6bca5214f13f5aede4fc8"}
Jul 14 01:00:40.637088 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:40.637061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:40.637254 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:40.637165    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:41.636871 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:41.636847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:41.637377 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:41.636882    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:41.637377 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:41.636957    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:41.637377 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:41.637118    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:41.762781 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:41.762725    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccc2deb6-ef0c-4606-9e80-2287b626ee13" containerID="72ccaf54fa1e80ad086562ae60ad546c864668a5ef87633d83c0a4d6cc5bfcaa" exitCode=0
Jul 14 01:00:41.762781 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:41.762777    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" event={"ID":"ccc2deb6-ef0c-4606-9e80-2287b626ee13","Type":"ContainerDied","Data":"72ccaf54fa1e80ad086562ae60ad546c864668a5ef87633d83c0a4d6cc5bfcaa"}
Jul 14 01:00:42.636856 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:42.636822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:42.637024 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:42.636939    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:43.220507 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:43.220475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:43.220676 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:43.220511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:43.220676 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.220637    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:43.220769 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.220714    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:01:15.22069251 +0000 UTC m=+66.115543135 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 01:00:43.220769 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.220637    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:43.220859 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.220801    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret podName:6d6a1433-d511-4afe-9860-ca3ca555fe54 nodeName:}" failed. No retries permitted until 2026-07-14 01:01:15.220781226 +0000 UTC m=+66.115631858 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret") pod "global-pull-secret-syncer-lwmb9" (UID: "6d6a1433-d511-4afe-9860-ca3ca555fe54") : object "kube-system"/"original-pull-secret" not registered
Jul 14 01:00:43.320965 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:43.320933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:43.321142 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.321094    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 01:00:43.321142 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.321118    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 01:00:43.321142 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.321131    2580 projected.go:196] Error preparing data for projected volume kube-api-access-9gnb6 for pod openshift-network-diagnostics/network-check-target-kzbkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:43.321294 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.321199    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6 podName:73ae53aa-354f-40c4-8c08-855b02fe3b4c nodeName:}" failed. No retries permitted until 2026-07-14 01:01:15.321179876 +0000 UTC m=+66.216030511 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gnb6" (UniqueName: "kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6") pod "network-check-target-kzbkf" (UID: "73ae53aa-354f-40c4-8c08-855b02fe3b4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 01:00:43.637098 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:43.637022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:43.637504 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:43.637022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:43.637504 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.637141    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzbkf" podUID="73ae53aa-354f-40c4-8c08-855b02fe3b4c"
Jul 14 01:00:43.637504 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:43.637234    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lwmb9" podUID="6d6a1433-d511-4afe-9860-ca3ca555fe54"
Jul 14 01:00:44.637248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:44.637006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:44.637631 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:44.637375    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:00:44.945653 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:44.945624    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-32.ec2.internal" event="NodeReady"
Jul 14 01:00:44.945837 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:44.945762    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 01:00:45.016714 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.016683    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-64wff"]
Jul 14 01:00:45.019258 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.019235    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f6wqv"]
Jul 14 01:00:45.019413 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.019386    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.021212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.021191    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f6wqv"
Jul 14 01:00:45.021313 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.021298    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 01:00:45.021377 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.021317    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jf2dk\""
Jul 14 01:00:45.021758 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.021743    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 01:00:45.023511 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.023492    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 01:00:45.023914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.023897    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 01:00:45.024224 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.024205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qwwjg\""
Jul 14 01:00:45.024301 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.024239    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 01:00:45.039783 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.036883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f6wqv"]
Jul 14 01:00:45.039783 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.036980    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-64wff"]
Jul 14 01:00:45.135763 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.135732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.135914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.135774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f-cert\") pod \"ingress-canary-f6wqv\" (UID: \"ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f\") " pod="openshift-ingress-canary/ingress-canary-f6wqv"
Jul 14 01:00:45.135914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.135804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hhkq\" (UniqueName: \"kubernetes.io/projected/ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f-kube-api-access-2hhkq\") pod \"ingress-canary-f6wqv\" (UID: \"ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f\") " pod="openshift-ingress-canary/ingress-canary-f6wqv"
Jul 14 01:00:45.135914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.135846    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-config-volume\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.135914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.135907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsw5q\" (UniqueName: \"kubernetes.io/projected/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-kube-api-access-fsw5q\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.136093 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.135959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-tmp-dir\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.237176 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.237089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.237176 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.237159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f-cert\") pod \"ingress-canary-f6wqv\" (UID: \"ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f\") " pod="openshift-ingress-canary/ingress-canary-f6wqv"
Jul 14 01:00:45.237350 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.237184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hhkq\" (UniqueName: \"kubernetes.io/projected/ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f-kube-api-access-2hhkq\") pod \"ingress-canary-f6wqv\" (UID: \"ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f\") " pod="openshift-ingress-canary/ingress-canary-f6wqv"
Jul 14 01:00:45.237350 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.237202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-config-volume\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.237350 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.237220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsw5q\" (UniqueName: \"kubernetes.io/projected/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-kube-api-access-fsw5q\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.237350 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.237240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-tmp-dir\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.237806 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:45.237237    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:00:45.237947 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:45.237892    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:00:45.737866649 +0000 UTC m=+36.632717280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:00:45.237947 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.237889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-tmp-dir\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.238069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.237952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-config-volume\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.242673 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.242654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f-cert\") pod \"ingress-canary-f6wqv\" (UID: \"ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f\") " pod="openshift-ingress-canary/ingress-canary-f6wqv"
Jul 14 01:00:45.247279 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.247252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsw5q\" (UniqueName: \"kubernetes.io/projected/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-kube-api-access-fsw5q\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.247534 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.247503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hhkq\" (UniqueName: \"kubernetes.io/projected/ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f-kube-api-access-2hhkq\") pod \"ingress-canary-f6wqv\" (UID: \"ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f\") " pod="openshift-ingress-canary/ingress-canary-f6wqv"
Jul 14 01:00:45.340769 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.340727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f6wqv"
Jul 14 01:00:45.636955 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.636880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:00:45.637102 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.637077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:00:45.638998 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.638970    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 01:00:45.639373 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.639047    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 01:00:45.639373 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.639090    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 01:00:45.639373 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.639152    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z2php\""
Jul 14 01:00:45.740257 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:45.740223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:45.740402 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:45.740367    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:00:45.740455 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:45.740438    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:00:46.74041805 +0000 UTC m=+37.635268692 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:00:46.195271 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:46.195248    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f6wqv"]
Jul 14 01:00:46.200822 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:00:46.200794    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded40dc6e_0146_4e9f_a1d5_6e92d671ca0f.slice/crio-080f437960a86e21aa99c8c7538657301ce88749446eb335180357517d848528 WatchSource:0}: Error finding container 080f437960a86e21aa99c8c7538657301ce88749446eb335180357517d848528: Status 404 returned error can't find the container with id 080f437960a86e21aa99c8c7538657301ce88749446eb335180357517d848528
Jul 14 01:00:46.636794 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:46.636771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:00:46.638813 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:46.638792    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6f2s8\""
Jul 14 01:00:46.638921 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:46.638819    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 01:00:46.748901 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:46.748882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:46.749156 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:46.749000    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:00:46.749156 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:46.749059    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:00:48.749043055 +0000 UTC m=+39.643893689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:00:46.776423 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:46.776262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f6wqv" event={"ID":"ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f","Type":"ContainerStarted","Data":"080f437960a86e21aa99c8c7538657301ce88749446eb335180357517d848528"}
Jul 14 01:00:46.779013 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:46.778993    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccc2deb6-ef0c-4606-9e80-2287b626ee13" containerID="4ca1cb4f5b52a1ab914c3da22cf9504b1362fe8ce35032289172bb26ccc3c0e4" exitCode=0
Jul 14 01:00:46.779097 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:46.779021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" event={"ID":"ccc2deb6-ef0c-4606-9e80-2287b626ee13","Type":"ContainerDied","Data":"4ca1cb4f5b52a1ab914c3da22cf9504b1362fe8ce35032289172bb26ccc3c0e4"}
Jul 14 01:00:47.785948 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:47.785912    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccc2deb6-ef0c-4606-9e80-2287b626ee13" containerID="76435d7e6d5e637470a0bc1da16683c8c869b4d7f2c77c330cb071b33eb742cb" exitCode=0
Jul 14 01:00:47.786378 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:47.785968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" event={"ID":"ccc2deb6-ef0c-4606-9e80-2287b626ee13","Type":"ContainerDied","Data":"76435d7e6d5e637470a0bc1da16683c8c869b4d7f2c77c330cb071b33eb742cb"}
Jul 14 01:00:48.764073 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:48.764040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:48.764240 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:48.764181    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:00:48.764312 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:48.764253    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:00:52.764230831 +0000 UTC m=+43.659081448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:00:48.790995 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:48.790966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" event={"ID":"ccc2deb6-ef0c-4606-9e80-2287b626ee13","Type":"ContainerStarted","Data":"e2c91c480cbbecd9f8cc6c57694ef71d1e81746b50b1f9484528f7bc3dbab4cd"}
Jul 14 01:00:48.812770 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:48.812729    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ksnzm" podStartSLOduration=8.251551294 podStartE2EDuration="39.812715518s" totalImagesPullingTime="31.561164224s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:48.812410642 +0000 UTC m=+39.707261295" watchObservedRunningTime="2026-07-14 01:00:48.812715518 +0000 UTC m=+39.707566157"
Jul 14 01:00:49.794462 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:49.794425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f6wqv" event={"ID":"ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f","Type":"ContainerStarted","Data":"680cfde8a5e486c6ddc4e1f1ab77f2c0b12783d47daab716448d5deae970f36d"}
Jul 14 01:00:49.810892 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:49.810837    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f6wqv" podStartSLOduration=2.847416641 podStartE2EDuration="5.810816953s" totalImagesPullingTime="2.963400312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:49.810619254 +0000 UTC m=+40.705469894" watchObservedRunningTime="2026-07-14 01:00:49.810816953 +0000 UTC m=+40.705667596"
Jul 14 01:00:52.788630 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:00:52.788594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:00:52.789064 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:52.788730    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:00:52.789064 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:00:52.788807    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:01:00.788787542 +0000 UTC m=+51.683638158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:01:00.840673 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:00.840635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:01:00.841107 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:01:00.840771    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:01:00.841107 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:01:00.840830    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:01:16.840811672 +0000 UTC m=+67.735662289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:01:10.770250 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:10.770225    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v5tc5"
Jul 14 01:01:15.238362 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.238329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:01:15.238362 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.238363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:01:15.240673 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.240652    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 01:01:15.240742 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.240727    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 01:01:15.248987 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:01:15.248961    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 14 01:01:15.249045 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:01:15.249033    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:02:19.249014842 +0000 UTC m=+130.143865463 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : secret "metrics-daemon-secret" not found
Jul 14 01:01:15.252265 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.252245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d6a1433-d511-4afe-9860-ca3ca555fe54-original-pull-secret\") pod \"global-pull-secret-syncer-lwmb9\" (UID: \"6d6a1433-d511-4afe-9860-ca3ca555fe54\") " pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:01:15.338857 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.338832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:01:15.340480 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.340463    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 01:01:15.347896 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.347877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lwmb9"
Jul 14 01:01:15.351399 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.351381    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 01:01:15.363759 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.363728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gnb6\" (UniqueName: \"kubernetes.io/projected/73ae53aa-354f-40c4-8c08-855b02fe3b4c-kube-api-access-9gnb6\") pod \"network-check-target-kzbkf\" (UID: \"73ae53aa-354f-40c4-8c08-855b02fe3b4c\") " pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:01:15.465259 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.465233    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lwmb9"]
Jul 14 01:01:15.469104 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:01:15.469074    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d6a1433_d511_4afe_9860_ca3ca555fe54.slice/crio-34971f546f8f32c9e02cb1577fc3e8b794a42ef2a8ccb35c6853dbff5ed2aae2 WatchSource:0}: Error finding container 34971f546f8f32c9e02cb1577fc3e8b794a42ef2a8ccb35c6853dbff5ed2aae2: Status 404 returned error can't find the container with id 34971f546f8f32c9e02cb1577fc3e8b794a42ef2a8ccb35c6853dbff5ed2aae2
Jul 14 01:01:15.656708 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.656684    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z2php\""
Jul 14 01:01:15.665086 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.665065    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:01:15.782442 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.782374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kzbkf"]
Jul 14 01:01:15.786003 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:01:15.785976    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ae53aa_354f_40c4_8c08_855b02fe3b4c.slice/crio-93b633b1b3b4c6af096bf9b8ea28e45050f6f744d94b2149fcf0fa4540262b8c WatchSource:0}: Error finding container 93b633b1b3b4c6af096bf9b8ea28e45050f6f744d94b2149fcf0fa4540262b8c: Status 404 returned error can't find the container with id 93b633b1b3b4c6af096bf9b8ea28e45050f6f744d94b2149fcf0fa4540262b8c
Jul 14 01:01:15.849965 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.849938    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kzbkf" event={"ID":"73ae53aa-354f-40c4-8c08-855b02fe3b4c","Type":"ContainerStarted","Data":"93b633b1b3b4c6af096bf9b8ea28e45050f6f744d94b2149fcf0fa4540262b8c"}
Jul 14 01:01:15.851006 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:15.850983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lwmb9" event={"ID":"6d6a1433-d511-4afe-9860-ca3ca555fe54","Type":"ContainerStarted","Data":"34971f546f8f32c9e02cb1577fc3e8b794a42ef2a8ccb35c6853dbff5ed2aae2"}
Jul 14 01:01:16.851336 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:16.851287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:01:16.851767 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:01:16.851468    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:01:16.851767 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:01:16.851572    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:01:48.851549402 +0000 UTC m=+99.746400022 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:01:19.862674 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:19.862633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kzbkf" event={"ID":"73ae53aa-354f-40c4-8c08-855b02fe3b4c","Type":"ContainerStarted","Data":"3287338bf686c31a12463d1c001a5d7e0c49082eadfa6406c6838d0cb7373066"}
Jul 14 01:01:19.863082 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:19.862725    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:01:21.869751 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:21.869714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lwmb9" event={"ID":"6d6a1433-d511-4afe-9860-ca3ca555fe54","Type":"ContainerStarted","Data":"026de5a16c115298d04da1c749c29201c408c12273a885baa8038ae9e9945c20"}
Jul 14 01:01:21.882832 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:21.882789    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kzbkf" podStartSLOduration=69.346929549 podStartE2EDuration="1m12.882775346s" totalImagesPullingTime="3.535845797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:01:19.877387184 +0000 UTC m=+70.772237824" watchObservedRunningTime="2026-07-14 01:01:21.882775346 +0000 UTC m=+72.777625963"
Jul 14 01:01:21.883181 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:21.883161    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lwmb9" podStartSLOduration=66.676769716 podStartE2EDuration="1m12.883154123s" totalImagesPullingTime="6.206384407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:01:21.882255674 +0000 UTC m=+72.777106314" watchObservedRunningTime="2026-07-14 01:01:21.883154123 +0000 UTC m=+72.778004761"
Jul 14 01:01:48.858768 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:48.858661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:01:48.859112 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:01:48.858771    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:01:48.859112 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:01:48.858831    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:02:52.858818254 +0000 UTC m=+163.753668885 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:01:50.867622 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:01:50.867591    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kzbkf"
Jul 14 01:02:02.621171 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.621140    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"]
Jul 14 01:02:02.623157 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.623141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"
Jul 14 01:02:02.625091 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.625060    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-s75fh\""
Jul 14 01:02:02.625091 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.625086    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 14 01:02:02.625622 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.625604    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 14 01:02:02.625727 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.625612    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 14 01:02:02.625925 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.625909    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 14 01:02:02.630301 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.630280    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"]
Jul 14 01:02:02.749695 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.749668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w5r8\" (UniqueName: \"kubernetes.io/projected/81cab238-19fe-4677-8643-e2ac58e6db64-kube-api-access-9w5r8\") pod \"managed-serviceaccount-addon-agent-74785d7fdd-c5m4g\" (UID: \"81cab238-19fe-4677-8643-e2ac58e6db64\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"
Jul 14 01:02:02.749804 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.749700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/81cab238-19fe-4677-8643-e2ac58e6db64-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74785d7fdd-c5m4g\" (UID: \"81cab238-19fe-4677-8643-e2ac58e6db64\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"
Jul 14 01:02:02.850127 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.850104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w5r8\" (UniqueName: \"kubernetes.io/projected/81cab238-19fe-4677-8643-e2ac58e6db64-kube-api-access-9w5r8\") pod \"managed-serviceaccount-addon-agent-74785d7fdd-c5m4g\" (UID: \"81cab238-19fe-4677-8643-e2ac58e6db64\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"
Jul 14 01:02:02.850200 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.850132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/81cab238-19fe-4677-8643-e2ac58e6db64-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74785d7fdd-c5m4g\" (UID: \"81cab238-19fe-4677-8643-e2ac58e6db64\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"
Jul 14 01:02:02.852513 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.852493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/81cab238-19fe-4677-8643-e2ac58e6db64-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74785d7fdd-c5m4g\" (UID: \"81cab238-19fe-4677-8643-e2ac58e6db64\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"
Jul 14 01:02:02.858236 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.858216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w5r8\" (UniqueName: \"kubernetes.io/projected/81cab238-19fe-4677-8643-e2ac58e6db64-kube-api-access-9w5r8\") pod \"managed-serviceaccount-addon-agent-74785d7fdd-c5m4g\" (UID: \"81cab238-19fe-4677-8643-e2ac58e6db64\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"
Jul 14 01:02:02.945509 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:02.945465    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"
Jul 14 01:02:03.061872 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:03.061844    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g"]
Jul 14 01:02:03.065660 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:02:03.065633    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81cab238_19fe_4677_8643_e2ac58e6db64.slice/crio-4ca21980f8ae698a4992ecd7f64899f088f67dfee5ca1050c2508783d7b6da10 WatchSource:0}: Error finding container 4ca21980f8ae698a4992ecd7f64899f088f67dfee5ca1050c2508783d7b6da10: Status 404 returned error can't find the container with id 4ca21980f8ae698a4992ecd7f64899f088f67dfee5ca1050c2508783d7b6da10
Jul 14 01:02:03.963439 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:03.963406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" event={"ID":"81cab238-19fe-4677-8643-e2ac58e6db64","Type":"ContainerStarted","Data":"4ca21980f8ae698a4992ecd7f64899f088f67dfee5ca1050c2508783d7b6da10"}
Jul 14 01:02:19.250230 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:19.250190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:02:19.250696 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:19.250299    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 14 01:02:19.250696 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:19.250357    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs podName:e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab nodeName:}" failed. No retries permitted until 2026-07-14 01:04:21.250344035 +0000 UTC m=+252.145194652 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs") pod "network-metrics-daemon-x2jvq" (UID: "e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab") : secret "metrics-daemon-secret" not found
Jul 14 01:02:45.219725 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:45.219680    2580 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway" image="registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72"
Jul 14 01:02:45.220113 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:45.219793    2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container addon-agent start failed in pod managed-serviceaccount-addon-agent-74785d7fdd-c5m4g_open-cluster-management-agent-addon(81cab238-19fe-4677-8643-e2ac58e6db64): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway" logger="UnhandledError"
Jul 14 01:02:45.220113 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:45.219832    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:02:46.061775 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:46.061737    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:02:48.031367 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:48.031318    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-64wff" podUID="5921a40f-30cb-4ca3-82ec-ce0fd64d96ad"
Jul 14 01:02:48.065298 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:48.065270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-64wff"
Jul 14 01:02:49.644378 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:49.644340    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-x2jvq" podUID="e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab"
Jul 14 01:02:52.867490 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:02:52.867456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:02:52.867850 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:52.867614    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 01:02:52.867850 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:52.867685    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls podName:5921a40f-30cb-4ca3-82ec-ce0fd64d96ad nodeName:}" failed. No retries permitted until 2026-07-14 01:04:54.867668207 +0000 UTC m=+285.762518827 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls") pod "dns-default-64wff" (UID: "5921a40f-30cb-4ca3-82ec-ce0fd64d96ad") : secret "dns-default-metrics-tls" not found
Jul 14 01:02:58.196864 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:58.196823    2580 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway" image="registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72"
Jul 14 01:02:58.197264 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:58.196932    2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container addon-agent start failed in pod managed-serviceaccount-addon-agent-74785d7fdd-c5m4g_open-cluster-management-agent-addon(81cab238-19fe-4677-8643-e2ac58e6db64): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway" logger="UnhandledError"
Jul 14 01:02:58.197264 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:02:58.196979    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:03:00.637028 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:00.636952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:03:10.637540 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:10.637497    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:03:25.968508 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:25.968469    2580 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway" image="registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72"
Jul 14 01:03:25.968922 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:25.968596    2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container addon-agent start failed in pod managed-serviceaccount-addon-agent-74785d7fdd-c5m4g_open-cluster-management-agent-addon(81cab238-19fe-4677-8643-e2ac58e6db64): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway" logger="UnhandledError"
Jul 14 01:03:25.968922 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:25.968635    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:03:40.638363 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:40.638318    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:03:47.500678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.500640    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t"]
Jul 14 01:03:47.502532 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.502507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t"
Jul 14 01:03:47.503848 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.503826    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"]
Jul 14 01:03:47.504358 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.504340    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 14 01:03:47.504557 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.504539    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 14 01:03:47.504919 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.504902    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5fr55\""
Jul 14 01:03:47.505460 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.505442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:47.507229 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.507211    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 14 01:03:47.507229 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.507221    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pnsnb\""
Jul 14 01:03:47.507370 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.507236    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 14 01:03:47.507370 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.507211    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 14 01:03:47.510308 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.510290    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t"]
Jul 14 01:03:47.522026 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.522005    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"]
Jul 14 01:03:47.606088 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.606057    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zg4pg"]
Jul 14 01:03:47.607794 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.607780    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"]
Jul 14 01:03:47.607918 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.607899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.609438 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.609419    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"]
Jul 14 01:03:47.609559 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.609533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.610012 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.609993    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 14 01:03:47.610208 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.610194    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 01:03:47.610468 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.610454    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 14 01:03:47.610721 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.610706    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6kshn\""
Jul 14 01:03:47.610994 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.610976    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 14 01:03:47.611750 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.611731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.611989 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.611917    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 01:03:47.611989 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.611966    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 14 01:03:47.612119 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.612056    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 01:03:47.612119 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.612057    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 01:03:47.612391 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.612371    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wpkf7\""
Jul 14 01:03:47.612619 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.612601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:47.612696 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.612627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbdjt\" (UniqueName: \"kubernetes.io/projected/c5908fd7-252d-4d02-809d-ca9e938a0975-kube-api-access-hbdjt\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:47.612696 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.612661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsvct\" (UniqueName: \"kubernetes.io/projected/4390937e-579c-4a5e-a8f5-98a19f02b652-kube-api-access-zsvct\") pod \"volume-data-source-validator-78495597f6-77k6t\" (UID: \"4390937e-579c-4a5e-a8f5-98a19f02b652\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t"
Jul 14 01:03:47.617959 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.617944    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 14 01:03:47.620294 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.620276    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 14 01:03:47.620384 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.620321    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 14 01:03:47.620636 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.620619    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-66lf6\""
Jul 14 01:03:47.621043 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.621025    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 14 01:03:47.628064 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.628045    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zg4pg"]
Jul 14 01:03:47.629131 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.629109    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"]
Jul 14 01:03:47.632207 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.632187    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 14 01:03:47.635917 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.635780    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"]
Jul 14 01:03:47.702540 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.702504    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"]
Jul 14 01:03:47.704329 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.704314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.706224 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.706201    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 14 01:03:47.706224 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.706220    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 14 01:03:47.706404 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.706246    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 14 01:03:47.706404 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.706261    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 14 01:03:47.706545 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.706504    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qjdlh\""
Jul 14 01:03:47.706957 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.706941    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66789c5848-94z9l"]
Jul 14 01:03:47.708765 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.708749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.710627 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.710608    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 14 01:03:47.710702 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.710614    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 14 01:03:47.710702 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.710614    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 14 01:03:47.710702 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.710617    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5xx4b\""
Jul 14 01:03:47.712985 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.712962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.713060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.713122 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k965c\" (UniqueName: \"kubernetes.io/projected/239efbb9-39bb-439f-982a-5234ea1e10b5-kube-api-access-k965c\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.713189 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a40711fd-430c-4648-a9c8-5212e97e36b2-config\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.713189 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:47.713306 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbdjt\" (UniqueName: \"kubernetes.io/projected/c5908fd7-252d-4d02-809d-ca9e938a0975-kube-api-access-hbdjt\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:47.713306 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-tmp\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.713306 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.713306 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:47.713280    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 01:03:47.713306 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-snapshots\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.713590 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:47.713342    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls podName:c5908fd7-252d-4d02-809d-ca9e938a0975 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:48.213323999 +0000 UTC m=+219.108174631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls") pod "cluster-samples-operator-65db858894-hnjd5" (UID: "c5908fd7-252d-4d02-809d-ca9e938a0975") : secret "samples-operator-tls" not found
Jul 14 01:03:47.713590 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m85zg\" (UniqueName: \"kubernetes.io/projected/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-kube-api-access-m85zg\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.713590 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-serving-cert\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.713590 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a40711fd-430c-4648-a9c8-5212e97e36b2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.713590 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsvct\" (UniqueName: \"kubernetes.io/projected/4390937e-579c-4a5e-a8f5-98a19f02b652-kube-api-access-zsvct\") pod \"volume-data-source-validator-78495597f6-77k6t\" (UID: \"4390937e-579c-4a5e-a8f5-98a19f02b652\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t"
Jul 14 01:03:47.713590 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltlsd\" (UniqueName: \"kubernetes.io/projected/a40711fd-430c-4648-a9c8-5212e97e36b2-kube-api-access-ltlsd\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.713590 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.713551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/239efbb9-39bb-439f-982a-5234ea1e10b5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.715226 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.715205    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"]
Jul 14 01:03:47.716965 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.716945    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 14 01:03:47.721259 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.721236    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66789c5848-94z9l"]
Jul 14 01:03:47.727201 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.727174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsvct\" (UniqueName: \"kubernetes.io/projected/4390937e-579c-4a5e-a8f5-98a19f02b652-kube-api-access-zsvct\") pod \"volume-data-source-validator-78495597f6-77k6t\" (UID: \"4390937e-579c-4a5e-a8f5-98a19f02b652\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t"
Jul 14 01:03:47.727290 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.727276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbdjt\" (UniqueName: \"kubernetes.io/projected/c5908fd7-252d-4d02-809d-ca9e938a0975-kube-api-access-hbdjt\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:47.812966 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.812898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t"
Jul 14 01:03:47.814130 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.814185 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-tmp\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.814185 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/239efbb9-39bb-439f-982a-5234ea1e10b5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.814253 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-bound-sa-token\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.814253 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-serving-cert\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.814340 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.814340 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkkvz\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-kube-api-access-bkkvz\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.814340 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.814476 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-image-registry-private-configuration\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.814476 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:47.814415    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:47.814476 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.814648 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:47.814483    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls podName:239efbb9-39bb-439f-982a-5234ea1e10b5 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:48.314462093 +0000 UTC m=+219.209312712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wfqhk" (UID: "239efbb9-39bb-439f-982a-5234ea1e10b5") : secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:47.814648 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-tmp\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.814648 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k965c\" (UniqueName: \"kubernetes.io/projected/239efbb9-39bb-439f-982a-5234ea1e10b5-kube-api-access-k965c\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814648    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnv4r\" (UniqueName: \"kubernetes.io/projected/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-kube-api-access-pnv4r\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a40711fd-430c-4648-a9c8-5212e97e36b2-config\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-trusted-ca\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.814802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-snapshots\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m85zg\" (UniqueName: \"kubernetes.io/projected/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-kube-api-access-m85zg\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a40711fd-430c-4648-a9c8-5212e97e36b2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltlsd\" (UniqueName: \"kubernetes.io/projected/a40711fd-430c-4648-a9c8-5212e97e36b2-kube-api-access-ltlsd\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/239efbb9-39bb-439f-982a-5234ea1e10b5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815172    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f305f6e3-06c6-4680-a58e-51b371866bea-ca-trust-extracted\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-registry-certificates\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.815408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-installation-pull-secrets\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.816069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815722    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.816069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.816069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a40711fd-430c-4648-a9c8-5212e97e36b2-config\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.816069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.815991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-snapshots\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.816868 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.816851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-serving-cert\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.817452 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.817431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a40711fd-430c-4648-a9c8-5212e97e36b2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.822538 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.822497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k965c\" (UniqueName: \"kubernetes.io/projected/239efbb9-39bb-439f-982a-5234ea1e10b5-kube-api-access-k965c\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:47.825642 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.825621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltlsd\" (UniqueName: \"kubernetes.io/projected/a40711fd-430c-4648-a9c8-5212e97e36b2-kube-api-access-ltlsd\") pod \"service-ca-operator-7695fc6c8c-bn8jh\" (UID: \"a40711fd-430c-4648-a9c8-5212e97e36b2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.826451 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.826429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m85zg\" (UniqueName: \"kubernetes.io/projected/375faa1c-7017-4d2d-bcf4-b9db1a90c6b9-kube-api-access-m85zg\") pod \"insights-operator-68d9dd94cf-zg4pg\" (UID: \"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9\") " pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.916128 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-trusted-ca\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916273 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f305f6e3-06c6-4680-a58e-51b371866bea-ca-trust-extracted\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916273 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-registry-certificates\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916273 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-installation-pull-secrets\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916273 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.916273 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916267    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-bound-sa-token\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916546 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916546 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkvz\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-kube-api-access-bkkvz\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916546 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-image-registry-private-configuration\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916546 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnv4r\" (UniqueName: \"kubernetes.io/projected/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-kube-api-access-pnv4r\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.916546 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.916784 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f305f6e3-06c6-4680-a58e-51b371866bea-ca-trust-extracted\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.916911 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.916968 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.916924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-registry-certificates\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.917084 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:47.917062    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 01:03:47.917154 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:47.917089    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66789c5848-94z9l: secret "image-registry-tls" not found
Jul 14 01:03:47.917154 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:47.917150    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls podName:f305f6e3-06c6-4680-a58e-51b371866bea nodeName:}" failed. No retries permitted until 2026-07-14 01:03:48.417131362 +0000 UTC m=+219.311981993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls") pod "image-registry-66789c5848-94z9l" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea") : secret "image-registry-tls" not found
Jul 14 01:03:47.917686 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.917665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-trusted-ca\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.918278 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.918232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg"
Jul 14 01:03:47.919117 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.919086    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-installation-pull-secrets\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.919327 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.919305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.919416 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.919401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-image-registry-private-configuration\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.926896 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.926871    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnv4r\" (UniqueName: \"kubernetes.io/projected/aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb-kube-api-access-pnv4r\") pod \"kube-storage-version-migrator-operator-799cb7f958-vfd7q\" (UID: \"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:47.926983 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.926899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-bound-sa-token\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.927728 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.927710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkkvz\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-kube-api-access-bkkvz\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:47.929587 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.929567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"
Jul 14 01:03:47.936140 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:47.936106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t"]
Jul 14 01:03:47.941346 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:03:47.941325    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4390937e_579c_4a5e_a8f5_98a19f02b652.slice/crio-c1f320eb2d8c416dbd23dd225c514fda982d929789735957795fdb80c58b3748 WatchSource:0}: Error finding container c1f320eb2d8c416dbd23dd225c514fda982d929789735957795fdb80c58b3748: Status 404 returned error can't find the container with id c1f320eb2d8c416dbd23dd225c514fda982d929789735957795fdb80c58b3748
Jul 14 01:03:48.014826 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.014798    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"
Jul 14 01:03:48.048105 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.048073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zg4pg"]
Jul 14 01:03:48.052915 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:03:48.052891    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375faa1c_7017_4d2d_bcf4_b9db1a90c6b9.slice/crio-36d58217fb18a573f5e56d84cb2710217fd846fe9fca41296c415bc118009499 WatchSource:0}: Error finding container 36d58217fb18a573f5e56d84cb2710217fd846fe9fca41296c415bc118009499: Status 404 returned error can't find the container with id 36d58217fb18a573f5e56d84cb2710217fd846fe9fca41296c415bc118009499
Jul 14 01:03:48.064605 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.064390    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh"]
Jul 14 01:03:48.068990 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:03:48.068947    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40711fd_430c_4648_a9c8_5212e97e36b2.slice/crio-9afeeb2a6d441c31e5c7a809c8bfe62cc82edf9f37b4150431943568bb2e72bf WatchSource:0}: Error finding container 9afeeb2a6d441c31e5c7a809c8bfe62cc82edf9f37b4150431943568bb2e72bf: Status 404 returned error can't find the container with id 9afeeb2a6d441c31e5c7a809c8bfe62cc82edf9f37b4150431943568bb2e72bf
Jul 14 01:03:48.141535 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.141485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q"]
Jul 14 01:03:48.145376 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:03:48.145354    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa3ef89_02e7_4724_abf2_cc0c6fe7ebdb.slice/crio-6cb6aeaddda64816704990f0b9bfe8c4ede625a0278b2abda06b02b723a67a76 WatchSource:0}: Error finding container 6cb6aeaddda64816704990f0b9bfe8c4ede625a0278b2abda06b02b723a67a76: Status 404 returned error can't find the container with id 6cb6aeaddda64816704990f0b9bfe8c4ede625a0278b2abda06b02b723a67a76
Jul 14 01:03:48.202837 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.202802    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg" event={"ID":"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9","Type":"ContainerStarted","Data":"36d58217fb18a573f5e56d84cb2710217fd846fe9fca41296c415bc118009499"}
Jul 14 01:03:48.203780 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.203752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t" event={"ID":"4390937e-579c-4a5e-a8f5-98a19f02b652","Type":"ContainerStarted","Data":"c1f320eb2d8c416dbd23dd225c514fda982d929789735957795fdb80c58b3748"}
Jul 14 01:03:48.204837 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.204809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh" event={"ID":"a40711fd-430c-4648-a9c8-5212e97e36b2","Type":"ContainerStarted","Data":"9afeeb2a6d441c31e5c7a809c8bfe62cc82edf9f37b4150431943568bb2e72bf"}
Jul 14 01:03:48.205761 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.205742    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q" event={"ID":"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb","Type":"ContainerStarted","Data":"6cb6aeaddda64816704990f0b9bfe8c4ede625a0278b2abda06b02b723a67a76"}
Jul 14 01:03:48.219135 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.219116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:48.219240 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.219227    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 01:03:48.219289 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.219280    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls podName:c5908fd7-252d-4d02-809d-ca9e938a0975 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:49.219264202 +0000 UTC m=+220.114114819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls") pod "cluster-samples-operator-65db858894-hnjd5" (UID: "c5908fd7-252d-4d02-809d-ca9e938a0975") : secret "samples-operator-tls" not found
Jul 14 01:03:48.320306 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.320251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:48.320397 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.320382    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:48.320440 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.320433    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls podName:239efbb9-39bb-439f-982a-5234ea1e10b5 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:49.320419909 +0000 UTC m=+220.215270529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wfqhk" (UID: "239efbb9-39bb-439f-982a-5234ea1e10b5") : secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:48.420668 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:48.420647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:48.420768 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.420758    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 01:03:48.420812 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.420770    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66789c5848-94z9l: secret "image-registry-tls" not found
Jul 14 01:03:48.420812 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.420805    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls podName:f305f6e3-06c6-4680-a58e-51b371866bea nodeName:}" failed. No retries permitted until 2026-07-14 01:03:49.420795475 +0000 UTC m=+220.315646092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls") pod "image-registry-66789c5848-94z9l" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea") : secret "image-registry-tls" not found
Jul 14 01:03:48.726315 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.726239    2580 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d990c42fa092f11380846eb324e2d2a50dfb6f4548dd87c9768a87b6c8fbca48"
Jul 14 01:03:48.726730 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.726337    2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-storage-version-migrator-operator start failed in pod kube-storage-version-migrator-operator-799cb7f958-vfd7q_openshift-kube-storage-version-migrator-operator(aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError"
Jul 14 01:03:48.726730 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:48.726371    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-storage-version-migrator-operator\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q" podUID="aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb"
Jul 14 01:03:49.210248 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:49.210215    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-storage-version-migrator-operator\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d990c42fa092f11380846eb324e2d2a50dfb6f4548dd87c9768a87b6c8fbca48\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q" podUID="aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb"
Jul 14 01:03:49.227654 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:49.227383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:49.227654 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:49.227544    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 01:03:49.227654 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:49.227606    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls podName:c5908fd7-252d-4d02-809d-ca9e938a0975 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:51.227587366 +0000 UTC m=+222.122437989 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls") pod "cluster-samples-operator-65db858894-hnjd5" (UID: "c5908fd7-252d-4d02-809d-ca9e938a0975") : secret "samples-operator-tls" not found
Jul 14 01:03:49.328577 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:49.328547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:49.328684 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:49.328641    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:49.328747 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:49.328703    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls podName:239efbb9-39bb-439f-982a-5234ea1e10b5 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:51.328684214 +0000 UTC m=+222.223534851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wfqhk" (UID: "239efbb9-39bb-439f-982a-5234ea1e10b5") : secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:49.429588 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:49.429562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:49.429742 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:49.429722    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 01:03:49.429796 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:49.429744    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66789c5848-94z9l: secret "image-registry-tls" not found
Jul 14 01:03:49.429796 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:49.429793    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls podName:f305f6e3-06c6-4680-a58e-51b371866bea nodeName:}" failed. No retries permitted until 2026-07-14 01:03:51.429778294 +0000 UTC m=+222.324628911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls") pod "image-registry-66789c5848-94z9l" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea") : secret "image-registry-tls" not found
Jul 14 01:03:50.214576 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:50.214535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t" event={"ID":"4390937e-579c-4a5e-a8f5-98a19f02b652","Type":"ContainerStarted","Data":"0b368bf4b01bc0428fa8ed0a8afcde34abd51167ca3f13d8dc444dcb19aec680"}
Jul 14 01:03:50.228112 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:50.227940    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-77k6t" podStartSLOduration=1.8622917970000001 podStartE2EDuration="3.227924858s" totalImagesPullingTime="1.365633061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:03:50.226997413 +0000 UTC m=+221.121848051" watchObservedRunningTime="2026-07-14 01:03:50.227924858 +0000 UTC m=+221.122775498"
Jul 14 01:03:51.218482 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:51.218451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh" event={"ID":"a40711fd-430c-4648-a9c8-5212e97e36b2","Type":"ContainerStarted","Data":"9c667c9fa74c12ebadb663579c4ec27da3d6d80ce36b5c0d5b747b5e3aba2188"}
Jul 14 01:03:51.234085 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:51.234039    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh" podStartSLOduration=1.923032861 podStartE2EDuration="4.234025918s" totalImagesPullingTime="2.310993057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:03:51.233545661 +0000 UTC m=+222.128396299" watchObservedRunningTime="2026-07-14 01:03:51.234025918 +0000 UTC m=+222.128876557"
Jul 14 01:03:51.244420 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:51.244394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:51.244726 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:51.244704    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 01:03:51.244830 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:51.244771    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls podName:c5908fd7-252d-4d02-809d-ca9e938a0975 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:55.24475208 +0000 UTC m=+226.139602700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls") pod "cluster-samples-operator-65db858894-hnjd5" (UID: "c5908fd7-252d-4d02-809d-ca9e938a0975") : secret "samples-operator-tls" not found
Jul 14 01:03:51.345363 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:51.345330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:51.345563 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:51.345446    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:51.345563 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:51.345511    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls podName:239efbb9-39bb-439f-982a-5234ea1e10b5 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:55.345494081 +0000 UTC m=+226.240344698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wfqhk" (UID: "239efbb9-39bb-439f-982a-5234ea1e10b5") : secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:51.445843 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:51.445776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:51.445954 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:51.445910    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 01:03:51.445954 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:51.445925    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66789c5848-94z9l: secret "image-registry-tls" not found
Jul 14 01:03:51.446063 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:51.445979    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls podName:f305f6e3-06c6-4680-a58e-51b371866bea nodeName:}" failed. No retries permitted until 2026-07-14 01:03:55.445961217 +0000 UTC m=+226.340811854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls") pod "image-registry-66789c5848-94z9l" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea") : secret "image-registry-tls" not found
Jul 14 01:03:51.643486 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:51.643121    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:03:52.222727 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.222695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg" event={"ID":"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9","Type":"ContainerStarted","Data":"510b759ac40bbabaf1b4bcbb86d0ccf82264a55bf64bb4180617b6fd634faba3"}
Jul 14 01:03:52.237159 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.237111    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg" podStartSLOduration=2.112505456 podStartE2EDuration="5.237095512s" totalImagesPullingTime="3.124590056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:03:52.235688231 +0000 UTC m=+223.130538871" watchObservedRunningTime="2026-07-14 01:03:52.237095512 +0000 UTC m=+223.131946151"
Jul 14 01:03:52.885413 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.885384    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xjcvt"]
Jul 14 01:03:52.887495 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.887480    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:52.889609 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.889584    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfczl\""
Jul 14 01:03:52.889895 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.889614    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 01:03:52.890028 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.890012    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 01:03:52.897499 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.897477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xjcvt"]
Jul 14 01:03:52.955284 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.955259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/911ff649-c9b4-422e-a473-8213dfead665-data-volume\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:52.955409 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.955315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:52.955409 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.955359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:52.955409 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.955382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvt52\" (UniqueName: \"kubernetes.io/projected/911ff649-c9b4-422e-a473-8213dfead665-kube-api-access-hvt52\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:52.955601 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:52.955440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/911ff649-c9b4-422e-a473-8213dfead665-crio-socket\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.056592 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.056569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/911ff649-c9b4-422e-a473-8213dfead665-crio-socket\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.056733 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.056621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/911ff649-c9b4-422e-a473-8213dfead665-data-volume\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.056733 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.056711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/911ff649-c9b4-422e-a473-8213dfead665-crio-socket\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.056829 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.056787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.056829 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.056821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.056923 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.056840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt52\" (UniqueName: \"kubernetes.io/projected/911ff649-c9b4-422e-a473-8213dfead665-kube-api-access-hvt52\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.056923 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.056886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/911ff649-c9b4-422e-a473-8213dfead665-data-volume\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.057019 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:53.056946    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 01:03:53.057019 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:53.056971    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 01:03:53.057093 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:53.057021    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:53.557001482 +0000 UTC m=+224.451852101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : configmap "kube-rbac-proxy" not found
Jul 14 01:03:53.057093 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:53.057039    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:53.557030204 +0000 UTC m=+224.451880825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : secret "insights-runtime-extractor-tls" not found
Jul 14 01:03:53.064043 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.064022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvt52\" (UniqueName: \"kubernetes.io/projected/911ff649-c9b4-422e-a473-8213dfead665-kube-api-access-hvt52\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.560232 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.560192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.560232 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:53.560237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:53.560684 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:53.560325    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 01:03:53.560684 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:53.560324    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 01:03:53.560684 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:53.560370    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:54.560359112 +0000 UTC m=+225.455209730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : configmap "kube-rbac-proxy" not found
Jul 14 01:03:53.560684 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:53.560382    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:54.560376303 +0000 UTC m=+225.455226919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : secret "insights-runtime-extractor-tls" not found
Jul 14 01:03:54.096542 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.096495    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-5sp6n"]
Jul 14 01:03:54.098552 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.098516    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.100307 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.100282    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 14 01:03:54.100307 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.100301    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ks9dn\""
Jul 14 01:03:54.100535 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.100333    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 14 01:03:54.100535 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.100338    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 14 01:03:54.100740 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.100725    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 14 01:03:54.105923 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.105902    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-5sp6n"]
Jul 14 01:03:54.164754 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.164735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ae011d3c-1b11-466b-b7e9-9f782eed8310-signing-cabundle\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.164866 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.164787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ae011d3c-1b11-466b-b7e9-9f782eed8310-signing-key\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.164866 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.164805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z7g6\" (UniqueName: \"kubernetes.io/projected/ae011d3c-1b11-466b-b7e9-9f782eed8310-kube-api-access-9z7g6\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.265967 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.265941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ae011d3c-1b11-466b-b7e9-9f782eed8310-signing-cabundle\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.266060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.265999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ae011d3c-1b11-466b-b7e9-9f782eed8310-signing-key\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.266060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.266025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7g6\" (UniqueName: \"kubernetes.io/projected/ae011d3c-1b11-466b-b7e9-9f782eed8310-kube-api-access-9z7g6\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.266618 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.266600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ae011d3c-1b11-466b-b7e9-9f782eed8310-signing-cabundle\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.268378 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.268360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ae011d3c-1b11-466b-b7e9-9f782eed8310-signing-key\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.272692 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.272672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z7g6\" (UniqueName: \"kubernetes.io/projected/ae011d3c-1b11-466b-b7e9-9f782eed8310-kube-api-access-9z7g6\") pod \"service-ca-79f68b9f4f-5sp6n\" (UID: \"ae011d3c-1b11-466b-b7e9-9f782eed8310\") " pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.407737 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.407690    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n"
Jul 14 01:03:54.529725 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.529692    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-5sp6n"]
Jul 14 01:03:54.532601 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:03:54.532576    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae011d3c_1b11_466b_b7e9_9f782eed8310.slice/crio-29b3798da9c5fc53f43535eaa23ff583770ddcfa7818cea3e2367f2fb10c18f2 WatchSource:0}: Error finding container 29b3798da9c5fc53f43535eaa23ff583770ddcfa7818cea3e2367f2fb10c18f2: Status 404 returned error can't find the container with id 29b3798da9c5fc53f43535eaa23ff583770ddcfa7818cea3e2367f2fb10c18f2
Jul 14 01:03:54.568498 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.568475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:54.568831 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.568515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:54.569032 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:54.569009    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 01:03:54.569113 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:54.569040    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 01:03:54.569163 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:54.569131    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:56.56908688 +0000 UTC m=+227.463937511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : secret "insights-runtime-extractor-tls" not found
Jul 14 01:03:54.569299 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:54.569287    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:03:56.56926631 +0000 UTC m=+227.464116941 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : configmap "kube-rbac-proxy" not found
Jul 14 01:03:54.868635 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:54.868606    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ggkk8_e2073ac1-0532-4124-a3ef-d37024685bc1/dns-node-resolver/0.log"
Jul 14 01:03:55.233018 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:55.232929    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n" event={"ID":"ae011d3c-1b11-466b-b7e9-9f782eed8310","Type":"ContainerStarted","Data":"21b6aa28d178fc421c0f76d2cb989729ca45536b68cc04ecfc00a708edc121a4"}
Jul 14 01:03:55.233199 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:55.233027    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n" event={"ID":"ae011d3c-1b11-466b-b7e9-9f782eed8310","Type":"ContainerStarted","Data":"29b3798da9c5fc53f43535eaa23ff583770ddcfa7818cea3e2367f2fb10c18f2"}
Jul 14 01:03:55.274716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:55.274690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:03:55.274899 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:55.274868    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 01:03:55.274969 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:55.274957    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls podName:c5908fd7-252d-4d02-809d-ca9e938a0975 nodeName:}" failed. No retries permitted until 2026-07-14 01:04:03.274935816 +0000 UTC m=+234.169786446 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls") pod "cluster-samples-operator-65db858894-hnjd5" (UID: "c5908fd7-252d-4d02-809d-ca9e938a0975") : secret "samples-operator-tls" not found
Jul 14 01:03:55.375873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:55.375842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:03:55.376010 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:55.375992    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:55.376068 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:55.376057    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls podName:239efbb9-39bb-439f-982a-5234ea1e10b5 nodeName:}" failed. No retries permitted until 2026-07-14 01:04:03.376040911 +0000 UTC m=+234.270891530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wfqhk" (UID: "239efbb9-39bb-439f-982a-5234ea1e10b5") : secret "cluster-monitoring-operator-tls" not found
Jul 14 01:03:55.477161 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:55.477134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:03:55.477286 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:55.477270    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 14 01:03:55.477324 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:55.477288    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66789c5848-94z9l: secret "image-registry-tls" not found
Jul 14 01:03:55.477357 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:55.477339    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls podName:f305f6e3-06c6-4680-a58e-51b371866bea nodeName:}" failed. No retries permitted until 2026-07-14 01:04:03.477321708 +0000 UTC m=+234.372172331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls") pod "image-registry-66789c5848-94z9l" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea") : secret "image-registry-tls" not found
Jul 14 01:03:56.068712 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:56.068687    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lp74x_20434812-dd9a-43d5-8eff-612bbeff893d/node-ca/0.log"
Jul 14 01:03:56.468357 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:56.468295    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f6wqv_ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f/serve-healthcheck-canary/0.log"
Jul 14 01:03:56.585400 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:56.585366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:56.585557 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:03:56.585409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:03:56.585557 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:56.585495    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 01:03:56.585557 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:56.585497    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 01:03:56.585557 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:56.585559    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:04:00.585545496 +0000 UTC m=+231.480396113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : configmap "kube-rbac-proxy" not found
Jul 14 01:03:56.585706 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:03:56.585584    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:04:00.585566309 +0000 UTC m=+231.480416926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : secret "insights-runtime-extractor-tls" not found
Jul 14 01:04:00.618579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:00.618552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:04:00.618961 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:00.618586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:04:00.618961 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:00.618662    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 01:04:00.618961 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:00.618712    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:04:08.618698871 +0000 UTC m=+239.513549488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : configmap "kube-rbac-proxy" not found
Jul 14 01:04:00.618961 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:00.618714    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 01:04:00.618961 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:00.618788    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls podName:911ff649-c9b4-422e-a473-8213dfead665 nodeName:}" failed. No retries permitted until 2026-07-14 01:04:08.618770181 +0000 UTC m=+239.513620800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjcvt" (UID: "911ff649-c9b4-422e-a473-8213dfead665") : secret "insights-runtime-extractor-tls" not found
Jul 14 01:04:03.339032 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.339000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:04:03.341597 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.341565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5908fd7-252d-4d02-809d-ca9e938a0975-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hnjd5\" (UID: \"c5908fd7-252d-4d02-809d-ca9e938a0975\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:04:03.418536 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.418493    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"
Jul 14 01:04:03.439926 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.439900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:04:03.440038 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:03.440023    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 14 01:04:03.440095 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:03.440088    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls podName:239efbb9-39bb-439f-982a-5234ea1e10b5 nodeName:}" failed. No retries permitted until 2026-07-14 01:04:19.440070688 +0000 UTC m=+250.334921325 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wfqhk" (UID: "239efbb9-39bb-439f-982a-5234ea1e10b5") : secret "cluster-monitoring-operator-tls" not found
Jul 14 01:04:03.541033 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.541006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:04:03.543689 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.543664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"image-registry-66789c5848-94z9l\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") " pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:04:03.555930 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.555880    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-5sp6n" podStartSLOduration=9.555860311 podStartE2EDuration="9.555860311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:03:55.247608781 +0000 UTC m=+226.142459412" watchObservedRunningTime="2026-07-14 01:04:03.555860311 +0000 UTC m=+234.450710951"
Jul 14 01:04:03.556769 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.556752    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5"]
Jul 14 01:04:03.621144 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.621118    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:04:03.638281 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:03.638240    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:04:03.743426 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:03.743348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66789c5848-94z9l"]
Jul 14 01:04:03.747571 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:03.747547    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf305f6e3_06c6_4680_a58e_51b371866bea.slice/crio-ad1bf99decf0163b2af060ace87030cae81afd0932f80279993a4436281a4888 WatchSource:0}: Error finding container ad1bf99decf0163b2af060ace87030cae81afd0932f80279993a4436281a4888: Status 404 returned error can't find the container with id ad1bf99decf0163b2af060ace87030cae81afd0932f80279993a4436281a4888
Jul 14 01:04:04.261694 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:04.261657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66789c5848-94z9l" event={"ID":"f305f6e3-06c6-4680-a58e-51b371866bea","Type":"ContainerStarted","Data":"78a8bcdf3b6a6111a0ee2103f93673c5186feb2231ee5259276a7340fd30d9e7"}
Jul 14 01:04:04.261885 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:04.261719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66789c5848-94z9l" event={"ID":"f305f6e3-06c6-4680-a58e-51b371866bea","Type":"ContainerStarted","Data":"ad1bf99decf0163b2af060ace87030cae81afd0932f80279993a4436281a4888"}
Jul 14 01:04:04.261885 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:04.261836    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:04:04.263037 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:04.263008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5" event={"ID":"c5908fd7-252d-4d02-809d-ca9e938a0975","Type":"ContainerStarted","Data":"c1241f38c138de3486b0eea3020c6c4395b41989412fde5a866626ed326b624b"}
Jul 14 01:04:04.278915 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:04.278874    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66789c5848-94z9l" podStartSLOduration=17.278859798 podStartE2EDuration="17.278859798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:04.277579715 +0000 UTC m=+235.172430354" watchObservedRunningTime="2026-07-14 01:04:04.278859798 +0000 UTC m=+235.173710436"
Jul 14 01:04:06.272856 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:06.272812    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q" event={"ID":"aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb","Type":"ContainerStarted","Data":"5fe2b18b974ae9ef5f1a8715520748b8cd3c42a9a2c9219a7a63be9041da6325"}
Jul 14 01:04:06.274711 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:06.274683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5" event={"ID":"c5908fd7-252d-4d02-809d-ca9e938a0975","Type":"ContainerStarted","Data":"8708f2eff56cfec6b862ea56753ce4a22347005e1142b8373898ff488cd0b52b"}
Jul 14 01:04:06.274847 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:06.274746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5" event={"ID":"c5908fd7-252d-4d02-809d-ca9e938a0975","Type":"ContainerStarted","Data":"b6fe68a574d0aead4505d11fdd03fcbab5aba2560d313a989abbf51ce5c7aa0b"}
Jul 14 01:04:06.286486 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:06.286445    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vfd7q" podStartSLOduration=1.425713505 podStartE2EDuration="19.286432682s" totalImagesPullingTime="17.860719177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-14 01:04:06.28575373 +0000 UTC m=+237.180604371" watchObservedRunningTime="2026-07-14 01:04:06.286432682 +0000 UTC m=+237.181283320"
Jul 14 01:04:06.300832 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:06.300795    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hnjd5" podStartSLOduration=17.470779508 podStartE2EDuration="19.3007833s" totalImagesPullingTime="1.830003792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:06.299466123 +0000 UTC m=+237.194316763" watchObservedRunningTime="2026-07-14 01:04:06.3007833 +0000 UTC m=+237.195633938"
Jul 14 01:04:08.683491 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:08.683461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:04:08.683491 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:08.683498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:04:08.684002 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:08.683983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/911ff649-c9b4-422e-a473-8213dfead665-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:04:08.685961 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:08.685938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/911ff649-c9b4-422e-a473-8213dfead665-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjcvt\" (UID: \"911ff649-c9b4-422e-a473-8213dfead665\") " pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:04:08.796491 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:08.796463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xjcvt"
Jul 14 01:04:08.919308 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:08.919282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xjcvt"]
Jul 14 01:04:08.922854 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:08.922832    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911ff649_c9b4_422e_a473_8213dfead665.slice/crio-6d8d1a58c8587b90a7413b4d97f0aae5d2bcdada8743f0b51c642a27a26848bd WatchSource:0}: Error finding container 6d8d1a58c8587b90a7413b4d97f0aae5d2bcdada8743f0b51c642a27a26848bd: Status 404 returned error can't find the container with id 6d8d1a58c8587b90a7413b4d97f0aae5d2bcdada8743f0b51c642a27a26848bd
Jul 14 01:04:09.286895 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:09.286865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xjcvt" event={"ID":"911ff649-c9b4-422e-a473-8213dfead665","Type":"ContainerStarted","Data":"6d8d1a58c8587b90a7413b4d97f0aae5d2bcdada8743f0b51c642a27a26848bd"}
Jul 14 01:04:13.300815 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:13.300779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xjcvt" event={"ID":"911ff649-c9b4-422e-a473-8213dfead665","Type":"ContainerStarted","Data":"413eb5fedc4790c9a689331041ef5c673b8bec44165843e5581376f3151fc19f"}
Jul 14 01:04:14.252700 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:14.252660    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66789c5848-94z9l"]
Jul 14 01:04:14.307444 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:14.307406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xjcvt" event={"ID":"911ff649-c9b4-422e-a473-8213dfead665","Type":"ContainerStarted","Data":"b88d093461b82cd37353013f28caa3594faff216d3f4ebe9f4f752a5dbee99be"}
Jul 14 01:04:16.316622 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:16.316593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xjcvt" event={"ID":"911ff649-c9b4-422e-a473-8213dfead665","Type":"ContainerStarted","Data":"ece672657a26798db1e84113cfc5bc7f964109773aee78991f6050fe241affa0"}
Jul 14 01:04:16.332121 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:16.332072    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xjcvt" podStartSLOduration=17.386520138 podStartE2EDuration="24.332057063s" totalImagesPullingTime="6.945536925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:03:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:16.330560428 +0000 UTC m=+247.225411072" watchObservedRunningTime="2026-07-14 01:04:16.332057063 +0000 UTC m=+247.226907702"
Jul 14 01:04:18.016796 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:18.016748    2580 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway" image="registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72"
Jul 14 01:04:18.017181 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:18.016863    2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container addon-agent start failed in pod managed-serviceaccount-addon-agent-74785d7fdd-c5m4g_open-cluster-management-agent-addon(81cab238-19fe-4677-8643-e2ac58e6db64): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway" logger="UnhandledError"
Jul 14 01:04:18.017181 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:18.016901    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:04:19.464027 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:19.463983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:04:19.466404 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:19.466381    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/239efbb9-39bb-439f-982a-5234ea1e10b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wfqhk\" (UID: \"239efbb9-39bb-439f-982a-5234ea1e10b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:04:19.725986 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:19.725913    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wpkf7\""
Jul 14 01:04:19.734386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:19.734365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"
Jul 14 01:04:19.862214 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:19.862186    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk"]
Jul 14 01:04:19.865221 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:19.865192    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239efbb9_39bb_439f_982a_5234ea1e10b5.slice/crio-a0b5d2fc8609fb0e213f15544afde7c564d1346cf72e238c66db9cef39a8ec77 WatchSource:0}: Error finding container a0b5d2fc8609fb0e213f15544afde7c564d1346cf72e238c66db9cef39a8ec77: Status 404 returned error can't find the container with id a0b5d2fc8609fb0e213f15544afde7c564d1346cf72e238c66db9cef39a8ec77
Jul 14 01:04:20.330367 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:20.330336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk" event={"ID":"239efbb9-39bb-439f-982a-5234ea1e10b5","Type":"ContainerStarted","Data":"a0b5d2fc8609fb0e213f15544afde7c564d1346cf72e238c66db9cef39a8ec77"}
Jul 14 01:04:21.276581 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:21.276546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:04:21.279393 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:21.279362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab-metrics-certs\") pod \"network-metrics-daemon-x2jvq\" (UID: \"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab\") " pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:04:21.339664 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:21.339642    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6f2s8\""
Jul 14 01:04:21.348583 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:21.348567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2jvq"
Jul 14 01:04:21.484485 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:21.484453    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x2jvq"]
Jul 14 01:04:21.920443 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:21.920417    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode14d77c0_4cb8_4bf2_8ed6_f8ecd3e975ab.slice/crio-1b5fd9983d58ce5990f8010d7a42c07e0c9d7fa261d544b9ad5c9b14f115dd7c WatchSource:0}: Error finding container 1b5fd9983d58ce5990f8010d7a42c07e0c9d7fa261d544b9ad5c9b14f115dd7c: Status 404 returned error can't find the container with id 1b5fd9983d58ce5990f8010d7a42c07e0c9d7fa261d544b9ad5c9b14f115dd7c
Jul 14 01:04:22.337876 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:22.337844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk" event={"ID":"239efbb9-39bb-439f-982a-5234ea1e10b5","Type":"ContainerStarted","Data":"93f2896ef2fdda6c887eb42a2bcff69e46bb10f8770f955e1508e38294307670"}
Jul 14 01:04:22.339111 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:22.339087    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2jvq" event={"ID":"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab","Type":"ContainerStarted","Data":"1b5fd9983d58ce5990f8010d7a42c07e0c9d7fa261d544b9ad5c9b14f115dd7c"}
Jul 14 01:04:22.352332 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:22.352289    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wfqhk" podStartSLOduration=33.267456094 podStartE2EDuration="35.352275026s" totalImagesPullingTime="2.084818932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:22.351374723 +0000 UTC m=+253.246225363" watchObservedRunningTime="2026-07-14 01:04:22.352275026 +0000 UTC m=+253.247125665"
Jul 14 01:04:24.257769 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:24.257735    2580 patch_prober.go:28] interesting pod/image-registry-66789c5848-94z9l container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 01:04:24.258244 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:24.257791    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66789c5848-94z9l" podUID="f305f6e3-06c6-4680-a58e-51b371866bea" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 01:04:24.347744 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:24.347713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2jvq" event={"ID":"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab","Type":"ContainerStarted","Data":"6d31f86cb4d8e920d4f6a0600445868c2bb1bc2b3dec478a8546654f318cf172"}
Jul 14 01:04:24.347881 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:24.347766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2jvq" event={"ID":"e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab","Type":"ContainerStarted","Data":"4b25a592bbf0ae2775d23add69e43b4b883bda816f5e2eeb4e9bac233c612912"}
Jul 14 01:04:24.361797 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:24.361752    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x2jvq" podStartSLOduration=253.85218044 podStartE2EDuration="4m15.361733407s" totalImagesPullingTime="1.509552967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:24.360272043 +0000 UTC m=+255.255122682" watchObservedRunningTime="2026-07-14 01:04:24.361733407 +0000 UTC m=+255.256584046"
Jul 14 01:04:30.637732 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:30.637694    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:04:31.811580 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.811544    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"]
Jul 14 01:04:31.815211 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.815192    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:31.817664 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.817640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 14 01:04:31.817664 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.817654    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tzbzh\""
Jul 14 01:04:31.817843 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.817713    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 01:04:31.817843 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.817658    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 14 01:04:31.824256 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.824236    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"]
Jul 14 01:04:31.826663 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.826645    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"]
Jul 14 01:04:31.830143 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.830125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:31.831792 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.831744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 14 01:04:31.831895 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.831818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 14 01:04:31.832116 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.832100    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 14 01:04:31.832201 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.832182    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6dp4q\""
Jul 14 01:04:31.836742 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.836720    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mwt5d"]
Jul 14 01:04:31.840185 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.840166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"]
Jul 14 01:04:31.840321 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.840308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.842356 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.842111    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 01:04:31.842356 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.842194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s7dcf\""
Jul 14 01:04:31.842356 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.842214    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 01:04:31.842356 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.842114    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 01:04:31.953370 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-tls\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953370 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:31.953579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33876569-f023-4f8f-a8de-c9f65228b8d5-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:31.953579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85wfx\" (UniqueName: \"kubernetes.io/projected/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-api-access-85wfx\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:31.953579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9dmw\" (UniqueName: \"kubernetes.io/projected/33876569-f023-4f8f-a8de-c9f65228b8d5-kube-api-access-q9dmw\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:31.953579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:31.953579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:31.953579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kqzs\" (UniqueName: \"kubernetes.io/projected/06982999-377f-4b78-a62e-281524f12e4d-kube-api-access-9kqzs\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953579 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-wtmp\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06982999-377f-4b78-a62e-281524f12e4d-metrics-client-ca\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/33876569-f023-4f8f-a8de-c9f65228b8d5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-textfile\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-sys\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33876569-f023-4f8f-a8de-c9f65228b8d5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-root\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:31.953927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:31.953832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.054784 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.054757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-sys\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.054920 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.054795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.054920 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.054829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33876569-f023-4f8f-a8de-c9f65228b8d5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.054920 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.054853    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-sys\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.054920 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.054855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-root\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.055133 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.054919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.055133 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.054980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-tls\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.055133 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.054985    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-root\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.055133 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.055133 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33876569-f023-4f8f-a8de-c9f65228b8d5-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.055133 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85wfx\" (UniqueName: \"kubernetes.io/projected/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-api-access-85wfx\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9dmw\" (UniqueName: \"kubernetes.io/projected/33876569-f023-4f8f-a8de-c9f65228b8d5-kube-api-access-q9dmw\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kqzs\" (UniqueName: \"kubernetes.io/projected/06982999-377f-4b78-a62e-281524f12e4d-kube-api-access-9kqzs\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-wtmp\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06982999-377f-4b78-a62e-281524f12e4d-metrics-client-ca\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/33876569-f023-4f8f-a8de-c9f65228b8d5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.055430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-textfile\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.055892 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.055455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.056475 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.056454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33876569-f023-4f8f-a8de-c9f65228b8d5-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.057367 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.057343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.057945 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.057781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-wtmp\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.058673 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.058377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.058673 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.058403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-textfile\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.058673 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.058567    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.058867 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.058851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.058929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.058916    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06982999-377f-4b78-a62e-281524f12e4d-metrics-client-ca\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.059049 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.059015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.059156 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.059139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.059222 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.059201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/33876569-f023-4f8f-a8de-c9f65228b8d5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.059749 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.059721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.062224 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.062171    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06982999-377f-4b78-a62e-281524f12e4d-node-exporter-tls\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.063563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.063495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/33876569-f023-4f8f-a8de-c9f65228b8d5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.064363 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.064341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kqzs\" (UniqueName: \"kubernetes.io/projected/06982999-377f-4b78-a62e-281524f12e4d-kube-api-access-9kqzs\") pod \"node-exporter-mwt5d\" (UID: \"06982999-377f-4b78-a62e-281524f12e4d\") " pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.064731 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.064710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85wfx\" (UniqueName: \"kubernetes.io/projected/3c1caa54-d144-4f9a-b7b4-9792d5ab544d-kube-api-access-85wfx\") pod \"kube-state-metrics-785778ff74-wxxkz\" (UID: \"3c1caa54-d144-4f9a-b7b4-9792d5ab544d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.065481 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.065464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9dmw\" (UniqueName: \"kubernetes.io/projected/33876569-f023-4f8f-a8de-c9f65228b8d5-kube-api-access-q9dmw\") pod \"openshift-state-metrics-84bd68b896-5rkfp\" (UID: \"33876569-f023-4f8f-a8de-c9f65228b8d5\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.125319 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.125291    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"
Jul 14 01:04:32.141378 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.141354    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"
Jul 14 01:04:32.150807 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.150781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mwt5d"
Jul 14 01:04:32.163881 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:32.163853    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06982999_377f_4b78_a62e_281524f12e4d.slice/crio-186ee69ed5d3347afdca0bca5ba06e05cdc07f87ba4772abffd3ee4860f38a36 WatchSource:0}: Error finding container 186ee69ed5d3347afdca0bca5ba06e05cdc07f87ba4772abffd3ee4860f38a36: Status 404 returned error can't find the container with id 186ee69ed5d3347afdca0bca5ba06e05cdc07f87ba4772abffd3ee4860f38a36
Jul 14 01:04:32.280543 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.280456    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp"]
Jul 14 01:04:32.284498 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:32.284468    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33876569_f023_4f8f_a8de_c9f65228b8d5.slice/crio-d1eade2803fa85de8f49fd31e564ecf521b81c1ab66899bdcdd993acc1531e91 WatchSource:0}: Error finding container d1eade2803fa85de8f49fd31e564ecf521b81c1ab66899bdcdd993acc1531e91: Status 404 returned error can't find the container with id d1eade2803fa85de8f49fd31e564ecf521b81c1ab66899bdcdd993acc1531e91
Jul 14 01:04:32.299836 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.299813    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wxxkz"]
Jul 14 01:04:32.301621 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:32.301600    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1caa54_d144_4f9a_b7b4_9792d5ab544d.slice/crio-9299c4e2e42fd4419aaefd2a49a08286053ee4dd422d1435343bafc66ddb5468 WatchSource:0}: Error finding container 9299c4e2e42fd4419aaefd2a49a08286053ee4dd422d1435343bafc66ddb5468: Status 404 returned error can't find the container with id 9299c4e2e42fd4419aaefd2a49a08286053ee4dd422d1435343bafc66ddb5468
Jul 14 01:04:32.376028 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.375965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwt5d" event={"ID":"06982999-377f-4b78-a62e-281524f12e4d","Type":"ContainerStarted","Data":"186ee69ed5d3347afdca0bca5ba06e05cdc07f87ba4772abffd3ee4860f38a36"}
Jul 14 01:04:32.377409 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.377381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz" event={"ID":"3c1caa54-d144-4f9a-b7b4-9792d5ab544d","Type":"ContainerStarted","Data":"9299c4e2e42fd4419aaefd2a49a08286053ee4dd422d1435343bafc66ddb5468"}
Jul 14 01:04:32.379044 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.379020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp" event={"ID":"33876569-f023-4f8f-a8de-c9f65228b8d5","Type":"ContainerStarted","Data":"2599b88d779dbc3438113ed85700c7a36cfaca6919340cfd28fdbdbbcf9698fc"}
Jul 14 01:04:32.379124 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:32.379074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp" event={"ID":"33876569-f023-4f8f-a8de-c9f65228b8d5","Type":"ContainerStarted","Data":"d1eade2803fa85de8f49fd31e564ecf521b81c1ab66899bdcdd993acc1531e91"}
Jul 14 01:04:33.387097 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:33.386331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwt5d" event={"ID":"06982999-377f-4b78-a62e-281524f12e4d","Type":"ContainerStarted","Data":"7684443bc3e3b858cd1bdc69c76e7fd5d118be79c603ae0b035b249962a7fc78"}
Jul 14 01:04:33.390209 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:33.390178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp" event={"ID":"33876569-f023-4f8f-a8de-c9f65228b8d5","Type":"ContainerStarted","Data":"44beb779b7c1458d7e2e29a7f6d41b862fac8299f1c5407da2b3f62ccdd402c0"}
Jul 14 01:04:34.263584 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:34.263553    2580 patch_prober.go:28] interesting pod/image-registry-66789c5848-94z9l container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 14 01:04:34.263680 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:34.263600    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66789c5848-94z9l" podUID="f305f6e3-06c6-4680-a58e-51b371866bea" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 14 01:04:34.396074 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:34.395264    2580 generic.go:356] "Generic (PLEG): container finished" podID="06982999-377f-4b78-a62e-281524f12e4d" containerID="7684443bc3e3b858cd1bdc69c76e7fd5d118be79c603ae0b035b249962a7fc78" exitCode=0
Jul 14 01:04:34.396074 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:34.395402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwt5d" event={"ID":"06982999-377f-4b78-a62e-281524f12e4d","Type":"ContainerDied","Data":"7684443bc3e3b858cd1bdc69c76e7fd5d118be79c603ae0b035b249962a7fc78"}
Jul 14 01:04:34.398069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:34.398035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz" event={"ID":"3c1caa54-d144-4f9a-b7b4-9792d5ab544d","Type":"ContainerStarted","Data":"0f130a0e8bb91d8f4e73e64c4e73fd34579c4bbff0638dcdf3cbc65e55e8ee88"}
Jul 14 01:04:34.398189 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:34.398104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz" event={"ID":"3c1caa54-d144-4f9a-b7b4-9792d5ab544d","Type":"ContainerStarted","Data":"52fd865553e51ba7673ae89ab46c750354008a7e6bc2644930747bfb663da573"}
Jul 14 01:04:34.400891 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:34.400872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp" event={"ID":"33876569-f023-4f8f-a8de-c9f65228b8d5","Type":"ContainerStarted","Data":"3ca34a39a86681d9884b5886aceb14f2a803e6b99f22300656e10e0e234d49c5"}
Jul 14 01:04:34.431300 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:34.431245    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-5rkfp" podStartSLOduration=1.619092294 podStartE2EDuration="3.431225435s" totalImagesPullingTime="1.812133141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:34.430682878 +0000 UTC m=+265.325533518" watchObservedRunningTime="2026-07-14 01:04:34.431225435 +0000 UTC m=+265.326076074"
Jul 14 01:04:35.407041 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:35.407000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwt5d" event={"ID":"06982999-377f-4b78-a62e-281524f12e4d","Type":"ContainerStarted","Data":"a9d15d966c151e6578170dc0cceffc68323223034148b5b218da8678397a32a0"}
Jul 14 01:04:35.407468 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:35.407072    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwt5d" event={"ID":"06982999-377f-4b78-a62e-281524f12e4d","Type":"ContainerStarted","Data":"f9bbb318f9911318b08a9b6426f358aa77483ec53e1b97f88dbbf5fff4aed77a"}
Jul 14 01:04:35.408971 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:35.408949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz" event={"ID":"3c1caa54-d144-4f9a-b7b4-9792d5ab544d","Type":"ContainerStarted","Data":"7adb9133436590d9eb960a9e97bb1edf16c179c9ea34357f72031768b7c5f924"}
Jul 14 01:04:35.424852 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:35.424812    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mwt5d" podStartSLOduration=3.226435044 podStartE2EDuration="4.424798294s" totalImagesPullingTime="1.111597625s" totalInitContainerRuntime="86.765625ms" isStatefulPod=true podCreationTimestamp="2026-07-14 01:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:35.423120771 +0000 UTC m=+266.317971411" watchObservedRunningTime="2026-07-14 01:04:35.424798294 +0000 UTC m=+266.319648933"
Jul 14 01:04:35.439229 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:35.439182    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-wxxkz" podStartSLOduration=2.526143274 podStartE2EDuration="4.439170272s" totalImagesPullingTime="1.913026998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:35.437895453 +0000 UTC m=+266.332746092" watchObservedRunningTime="2026-07-14 01:04:35.439170272 +0000 UTC m=+266.334020923"
Jul 14 01:04:36.131348 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.129627    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"]
Jul 14 01:04:36.134173 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.134149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.136639 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.136617    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fhso04bhvvr4l\""
Jul 14 01:04:36.136923 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.136901    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 01:04:36.137027 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.136949    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 14 01:04:36.137027 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.136983    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xzk2g\""
Jul 14 01:04:36.137027 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.136953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 14 01:04:36.137027 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.136986    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 14 01:04:36.140908 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.140883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"]
Jul 14 01:04:36.293425 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.293399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-secret-metrics-server-tls\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.293588 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.293432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-secret-metrics-server-client-certs\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.293588 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.293458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-client-ca-bundle\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.293588 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.293542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc45df5b-775b-404e-a9d4-1dec3705980e-audit-log\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.293691 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.293591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f25z2\" (UniqueName: \"kubernetes.io/projected/dc45df5b-775b-404e-a9d4-1dec3705980e-kube-api-access-f25z2\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.293691 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.293668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc45df5b-775b-404e-a9d4-1dec3705980e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.293691 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.293686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc45df5b-775b-404e-a9d4-1dec3705980e-metrics-server-audit-profiles\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.394273 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.394214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f25z2\" (UniqueName: \"kubernetes.io/projected/dc45df5b-775b-404e-a9d4-1dec3705980e-kube-api-access-f25z2\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.394367 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.394274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc45df5b-775b-404e-a9d4-1dec3705980e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.394367 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.394290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc45df5b-775b-404e-a9d4-1dec3705980e-metrics-server-audit-profiles\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.394367 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.394309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-secret-metrics-server-tls\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.394367 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.394326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-secret-metrics-server-client-certs\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.394367 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.394344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-client-ca-bundle\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.394625 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.394384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc45df5b-775b-404e-a9d4-1dec3705980e-audit-log\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.394947 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.394913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc45df5b-775b-404e-a9d4-1dec3705980e-audit-log\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.395262 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.395116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc45df5b-775b-404e-a9d4-1dec3705980e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.395395 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.395294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc45df5b-775b-404e-a9d4-1dec3705980e-metrics-server-audit-profiles\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.397007 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.396984    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-client-ca-bundle\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.397078 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.397034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-secret-metrics-server-client-certs\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.397177 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.397156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc45df5b-775b-404e-a9d4-1dec3705980e-secret-metrics-server-tls\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.401130 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.401109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f25z2\" (UniqueName: \"kubernetes.io/projected/dc45df5b-775b-404e-a9d4-1dec3705980e-kube-api-access-f25z2\") pod \"metrics-server-86dd9c8cf7-4rj2b\" (UID: \"dc45df5b-775b-404e-a9d4-1dec3705980e\") " pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.444438 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.444411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:36.574156 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:36.574121    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"]
Jul 14 01:04:36.578221 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:36.578175    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc45df5b_775b_404e_a9d4_1dec3705980e.slice/crio-345fc549386b640def87f23a81ceca5fc8bd90f77e626c213f5cdf4c912cb519 WatchSource:0}: Error finding container 345fc549386b640def87f23a81ceca5fc8bd90f77e626c213f5cdf4c912cb519: Status 404 returned error can't find the container with id 345fc549386b640def87f23a81ceca5fc8bd90f77e626c213f5cdf4c912cb519
Jul 14 01:04:37.014922 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.014887    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-58bcf6b854-58977"]
Jul 14 01:04:37.018027 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.018008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.019982 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.019962    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 14 01:04:37.020143 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.020123    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 14 01:04:37.020227 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.020210    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 14 01:04:37.020284 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.020213    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 14 01:04:37.020336 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.020229    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 14 01:04:37.020336 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.020316    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gmkwh\""
Jul 14 01:04:37.025818 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.025509    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 14 01:04:37.029734 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.029715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58bcf6b854-58977"]
Jul 14 01:04:37.100482 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.100461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-federate-client-tls\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.100632 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.100486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.100632 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.100511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-serving-certs-ca-bundle\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.100632 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.100547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-telemeter-client-tls\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.100632 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.100592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-metrics-client-ca\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.100767 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.100646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk49n\" (UniqueName: \"kubernetes.io/projected/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-kube-api-access-nk49n\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.100767 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.100716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-secret-telemeter-client\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.100767 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.100746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.201815 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.201793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-federate-client-tls\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.201932 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.201822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.201932 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.201852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-serving-certs-ca-bundle\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.201932 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.201877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-telemeter-client-tls\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.201932 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.201899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-metrics-client-ca\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.202129 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.201940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk49n\" (UniqueName: \"kubernetes.io/projected/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-kube-api-access-nk49n\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.202129 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.201990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-secret-telemeter-client\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.202129 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.202029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.202755 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.202730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-serving-certs-ca-bundle\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.202854 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.202736    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-metrics-client-ca\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.202854 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.202805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.204588 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.204560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-federate-client-tls\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.204663 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.204612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-telemeter-client-tls\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.204707 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.204673    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.205210 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.205194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-secret-telemeter-client\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.208961 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.208945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk49n\" (UniqueName: \"kubernetes.io/projected/a0ed9655-9ae8-49b1-875d-919b17b9c3cf-kube-api-access-nk49n\") pod \"telemeter-client-58bcf6b854-58977\" (UID: \"a0ed9655-9ae8-49b1-875d-919b17b9c3cf\") " pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.328834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.328771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58bcf6b854-58977"
Jul 14 01:04:37.421716 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.421675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b" event={"ID":"dc45df5b-775b-404e-a9d4-1dec3705980e","Type":"ContainerStarted","Data":"345fc549386b640def87f23a81ceca5fc8bd90f77e626c213f5cdf4c912cb519"}
Jul 14 01:04:37.491734 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:37.491683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58bcf6b854-58977"]
Jul 14 01:04:37.497748 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:37.497720    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0ed9655_9ae8_49b1_875d_919b17b9c3cf.slice/crio-f74aa2fd42733e708b2b3d61d6b9cd5c85474d331a66d0c9adad04862d1d16cf WatchSource:0}: Error finding container f74aa2fd42733e708b2b3d61d6b9cd5c85474d331a66d0c9adad04862d1d16cf: Status 404 returned error can't find the container with id f74aa2fd42733e708b2b3d61d6b9cd5c85474d331a66d0c9adad04862d1d16cf
Jul 14 01:04:38.026539 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.026484    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:04:38.029625 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.029604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.031604 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.031582    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 01:04:38.032385 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.032365    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fv8oalj5f9kq0\""
Jul 14 01:04:38.033066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.032662    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 01:04:38.033066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.032672    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 01:04:38.033066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.032706    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 01:04:38.033066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.032720    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 01:04:38.033066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.032729    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 01:04:38.033066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.032723    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 01:04:38.033066 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.032770    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 01:04:38.033455 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.033400    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jjvpc\""
Jul 14 01:04:38.033455 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.033408    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 01:04:38.033608 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.033585    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 01:04:38.033749 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.033731    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 01:04:38.034452 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.034071    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 01:04:38.035914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.035779    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 01:04:38.040404 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.040371    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 01:04:38.046598 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.046572    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:04:38.109151 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-web-config\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109296 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109296 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109296 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109296 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109454 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-config-out\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109454 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109454 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109591 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109591 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109670 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109670 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109670 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-config\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109796 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109796 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109796 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109796 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9mwp\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-kube-api-access-h9mwp\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.109956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.109870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210558 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9mwp\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-kube-api-access-h9mwp\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-web-config\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-config-out\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.210963 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-config\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.211578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.210984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.211578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.211017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.211578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.211042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.211578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.211076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.211578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.211106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.211578 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.211560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.211881 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.211839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.211944 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.211887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.212593 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.212204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.212593 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.212479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.214312 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.214286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.215215 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.214666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-web-config\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.215215 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.215165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.215548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.215404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-config-out\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.215615 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.215504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.215901 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.215853    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.218195 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.218171    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-config\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.218691 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.218668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.219090 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.219069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.219179 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.219165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.219279 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.219228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.219279 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.219240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.219447 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.219286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.220050 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.220030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9mwp\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-kube-api-access-h9mwp\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.221198 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.221178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.342426 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.342356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:38.428477 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.428415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b" event={"ID":"dc45df5b-775b-404e-a9d4-1dec3705980e","Type":"ContainerStarted","Data":"7140e40795b867d27957268429a71832517ea350bf40e12b331494d9bf9cea30"}
Jul 14 01:04:38.430232 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.430201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bcf6b854-58977" event={"ID":"a0ed9655-9ae8-49b1-875d-919b17b9c3cf","Type":"ContainerStarted","Data":"f74aa2fd42733e708b2b3d61d6b9cd5c85474d331a66d0c9adad04862d1d16cf"}
Jul 14 01:04:38.447369 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.447325    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b" podStartSLOduration=0.677532332 podStartE2EDuration="2.447309374s" totalImagesPullingTime="1.769777042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:38.445787587 +0000 UTC m=+269.340638229" watchObservedRunningTime="2026-07-14 01:04:38.447309374 +0000 UTC m=+269.342160012"
Jul 14 01:04:38.490953 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:38.490923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:04:38.495415 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:38.495380    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9d622f_1143_4748_8a20_139e9095ccb5.slice/crio-dfc1f8ccb40116ce82d0f468657d3fad96d5ddc85cd3f77eb4cadb8870b3949c WatchSource:0}: Error finding container dfc1f8ccb40116ce82d0f468657d3fad96d5ddc85cd3f77eb4cadb8870b3949c: Status 404 returned error can't find the container with id dfc1f8ccb40116ce82d0f468657d3fad96d5ddc85cd3f77eb4cadb8870b3949c
Jul 14 01:04:39.276133 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:39.276064    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66789c5848-94z9l" podUID="f305f6e3-06c6-4680-a58e-51b371866bea" containerName="registry" containerID="cri-o://78a8bcdf3b6a6111a0ee2103f93673c5186feb2231ee5259276a7340fd30d9e7" gracePeriod=30
Jul 14 01:04:39.437154 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:39.436936    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerStarted","Data":"dfc1f8ccb40116ce82d0f468657d3fad96d5ddc85cd3f77eb4cadb8870b3949c"}
Jul 14 01:04:39.439112 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:39.439082    2580 generic.go:356] "Generic (PLEG): container finished" podID="f305f6e3-06c6-4680-a58e-51b371866bea" containerID="78a8bcdf3b6a6111a0ee2103f93673c5186feb2231ee5259276a7340fd30d9e7" exitCode=0
Jul 14 01:04:39.439282 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:39.439253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66789c5848-94z9l" event={"ID":"f305f6e3-06c6-4680-a58e-51b371866bea","Type":"ContainerDied","Data":"78a8bcdf3b6a6111a0ee2103f93673c5186feb2231ee5259276a7340fd30d9e7"}
Jul 14 01:04:39.962336 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:39.962303    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:04:40.129580 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.129546    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-image-registry-private-configuration\") pod \"f305f6e3-06c6-4680-a58e-51b371866bea\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") "
Jul 14 01:04:40.129750 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.129612    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-registry-certificates\") pod \"f305f6e3-06c6-4680-a58e-51b371866bea\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") "
Jul 14 01:04:40.129750 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.129636    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkkvz\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-kube-api-access-bkkvz\") pod \"f305f6e3-06c6-4680-a58e-51b371866bea\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") "
Jul 14 01:04:40.129750 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.129659    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-bound-sa-token\") pod \"f305f6e3-06c6-4680-a58e-51b371866bea\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") "
Jul 14 01:04:40.129750 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.129732    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-trusted-ca\") pod \"f305f6e3-06c6-4680-a58e-51b371866bea\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") "
Jul 14 01:04:40.129951 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.129760    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") pod \"f305f6e3-06c6-4680-a58e-51b371866bea\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") "
Jul 14 01:04:40.129951 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.129785    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-installation-pull-secrets\") pod \"f305f6e3-06c6-4680-a58e-51b371866bea\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") "
Jul 14 01:04:40.129951 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.129885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f305f6e3-06c6-4680-a58e-51b371866bea-ca-trust-extracted\") pod \"f305f6e3-06c6-4680-a58e-51b371866bea\" (UID: \"f305f6e3-06c6-4680-a58e-51b371866bea\") "
Jul 14 01:04:40.130095 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.130046    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f305f6e3-06c6-4680-a58e-51b371866bea" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:04:40.130990 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.130967    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-registry-certificates\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:04:40.132221 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.132164    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f305f6e3-06c6-4680-a58e-51b371866bea" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:04:40.133563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.132950    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-kube-api-access-bkkvz" (OuterVolumeSpecName: "kube-api-access-bkkvz") pod "f305f6e3-06c6-4680-a58e-51b371866bea" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea"). InnerVolumeSpecName "kube-api-access-bkkvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:04:40.133563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.132955    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f305f6e3-06c6-4680-a58e-51b371866bea" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:04:40.134803 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.134541    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f305f6e3-06c6-4680-a58e-51b371866bea" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:04:40.134803 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.134566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f305f6e3-06c6-4680-a58e-51b371866bea" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:04:40.138669 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.138324    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f305f6e3-06c6-4680-a58e-51b371866bea" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:04:40.146402 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.146371    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f305f6e3-06c6-4680-a58e-51b371866bea-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f305f6e3-06c6-4680-a58e-51b371866bea" (UID: "f305f6e3-06c6-4680-a58e-51b371866bea"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 01:04:40.232358 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.232298    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-image-registry-private-configuration\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:04:40.232358 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.232325    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkkvz\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-kube-api-access-bkkvz\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:04:40.232358 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.232340    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-bound-sa-token\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:04:40.232358 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.232354    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f305f6e3-06c6-4680-a58e-51b371866bea-trusted-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:04:40.232572 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.232366    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f305f6e3-06c6-4680-a58e-51b371866bea-registry-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:04:40.232572 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.232377    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f305f6e3-06c6-4680-a58e-51b371866bea-installation-pull-secrets\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:04:40.232572 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.232390    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f305f6e3-06c6-4680-a58e-51b371866bea-ca-trust-extracted\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:04:40.445421 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.445389    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerID="3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488" exitCode=0
Jul 14 01:04:40.445565 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.445507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerDied","Data":"3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488"}
Jul 14 01:04:40.447041 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.447015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66789c5848-94z9l" event={"ID":"f305f6e3-06c6-4680-a58e-51b371866bea","Type":"ContainerDied","Data":"ad1bf99decf0163b2af060ace87030cae81afd0932f80279993a4436281a4888"}
Jul 14 01:04:40.447156 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.447071    2580 scope.go:117] "RemoveContainer" containerID="78a8bcdf3b6a6111a0ee2103f93673c5186feb2231ee5259276a7340fd30d9e7"
Jul 14 01:04:40.447293 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.447231    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66789c5848-94z9l"
Jul 14 01:04:40.450391 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.450367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bcf6b854-58977" event={"ID":"a0ed9655-9ae8-49b1-875d-919b17b9c3cf","Type":"ContainerStarted","Data":"00fc6f795e6f905502438ab53a58d2a9711f9e5acf714cbeb0f47c62a8258191"}
Jul 14 01:04:40.450495 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.450415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bcf6b854-58977" event={"ID":"a0ed9655-9ae8-49b1-875d-919b17b9c3cf","Type":"ContainerStarted","Data":"ab2b8d54e7d817ab1782a20ac5b78f148fbc6808922e7ec8c1fbae776317293c"}
Jul 14 01:04:40.488312 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.488242    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66789c5848-94z9l"]
Jul 14 01:04:40.491447 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:40.491425    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66789c5848-94z9l"]
Jul 14 01:04:41.458492 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:41.458436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bcf6b854-58977" event={"ID":"a0ed9655-9ae8-49b1-875d-919b17b9c3cf","Type":"ContainerStarted","Data":"23b23b662e737848603d94216cfc10247d5df48352d0548cee6828c9f09af0fc"}
Jul 14 01:04:41.478864 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:41.478782    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-58bcf6b854-58977" podStartSLOduration=2.740392201 podStartE2EDuration="5.478763808s" totalImagesPullingTime="2.738371607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:41.477014304 +0000 UTC m=+272.371864943" watchObservedRunningTime="2026-07-14 01:04:41.478763808 +0000 UTC m=+272.373614448"
Jul 14 01:04:41.642699 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:41.642662    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f305f6e3-06c6-4680-a58e-51b371866bea" path="/var/lib/kubelet/pods/f305f6e3-06c6-4680-a58e-51b371866bea/volumes"
Jul 14 01:04:42.638172 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:42.638134    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:04:44.474633 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:44.474596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerStarted","Data":"a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248"}
Jul 14 01:04:44.475021 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:44.474665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerStarted","Data":"c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d"}
Jul 14 01:04:46.487563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:46.487468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerStarted","Data":"f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27"}
Jul 14 01:04:46.487563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:46.487547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerStarted","Data":"6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3"}
Jul 14 01:04:46.487975 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:46.487577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerStarted","Data":"67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff"}
Jul 14 01:04:46.487975 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:46.487606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerStarted","Data":"57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8"}
Jul 14 01:04:46.512032 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:46.511990    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.257191769 podStartE2EDuration="8.511976106s" totalImagesPullingTime="7.234710003s" totalInitContainerRuntime="20.074334ms" isStatefulPod=false podCreationTimestamp="2026-07-14 01:04:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:46.510865622 +0000 UTC m=+277.405716261" watchObservedRunningTime="2026-07-14 01:04:46.511976106 +0000 UTC m=+277.406826745"
Jul 14 01:04:48.343436 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:48.343400    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:04:51.066120 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:51.066082    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-64wff" podUID="5921a40f-30cb-4ca3-82ec-ce0fd64d96ad"
Jul 14 01:04:51.508821 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:51.508793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-64wff"
Jul 14 01:04:54.949739 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:54.949708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:04:54.952060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:54.952041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921a40f-30cb-4ca3-82ec-ce0fd64d96ad-metrics-tls\") pod \"dns-default-64wff\" (UID: \"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad\") " pod="openshift-dns/dns-default-64wff"
Jul 14 01:04:55.111797 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:55.111767    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jf2dk\""
Jul 14 01:04:55.120151 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:55.120125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-64wff"
Jul 14 01:04:55.262485 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:55.262458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-64wff"]
Jul 14 01:04:55.265562 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:04:55.265516    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5921a40f_30cb_4ca3_82ec_ce0fd64d96ad.slice/crio-a3a2fbb51ac1b579540f31cfdfdecc26d1cd659b4e72b8bf54dae7c32aeb28c1 WatchSource:0}: Error finding container a3a2fbb51ac1b579540f31cfdfdecc26d1cd659b4e72b8bf54dae7c32aeb28c1: Status 404 returned error can't find the container with id a3a2fbb51ac1b579540f31cfdfdecc26d1cd659b4e72b8bf54dae7c32aeb28c1
Jul 14 01:04:55.525626 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:55.525599    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64wff" event={"ID":"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad","Type":"ContainerStarted","Data":"a3a2fbb51ac1b579540f31cfdfdecc26d1cd659b4e72b8bf54dae7c32aeb28c1"}
Jul 14 01:04:56.444852 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:56.444819    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:56.445215 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:56.444903    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:04:57.535914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:57.535886    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64wff" event={"ID":"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad","Type":"ContainerStarted","Data":"baa2372f1eecbc48ecfc0ecdeba1a9e57c0e64057c70999e192266be826078a3"}
Jul 14 01:04:57.536244 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:57.535941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64wff" event={"ID":"5921a40f-30cb-4ca3-82ec-ce0fd64d96ad","Type":"ContainerStarted","Data":"ce4fc561cd69107a562fcc1bc6d5d79d48bdb936108f32415fcb3585017808bd"}
Jul 14 01:04:57.536244 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:57.535992    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-64wff"
Jul 14 01:04:57.552404 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:04:57.552342    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-64wff" podStartSLOduration=251.71100778 podStartE2EDuration="4m13.552323866s" totalImagesPullingTime="1.841316086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:04:57.550760955 +0000 UTC m=+288.445611596" watchObservedRunningTime="2026-07-14 01:04:57.552323866 +0000 UTC m=+288.447174506"
Jul 14 01:04:57.638043 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:04:57.638013    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:05:01.554176 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:01.554093    2580 generic.go:356] "Generic (PLEG): container finished" podID="a40711fd-430c-4648-a9c8-5212e97e36b2" containerID="9c667c9fa74c12ebadb663579c4ec27da3d6d80ce36b5c0d5b747b5e3aba2188" exitCode=0
Jul 14 01:05:01.554540 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:01.554165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh" event={"ID":"a40711fd-430c-4648-a9c8-5212e97e36b2","Type":"ContainerDied","Data":"9c667c9fa74c12ebadb663579c4ec27da3d6d80ce36b5c0d5b747b5e3aba2188"}
Jul 14 01:05:01.554540 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:01.554476    2580 scope.go:117] "RemoveContainer" containerID="9c667c9fa74c12ebadb663579c4ec27da3d6d80ce36b5c0d5b747b5e3aba2188"
Jul 14 01:05:02.560470 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:02.560435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bn8jh" event={"ID":"a40711fd-430c-4648-a9c8-5212e97e36b2","Type":"ContainerStarted","Data":"87623df2b0bc35df3a64d07aa5f75700d60c3d94f93d10c474f2a96d82c603d6"}
Jul 14 01:05:07.542077 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:07.542051    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-64wff"
Jul 14 01:05:09.494607 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:09.494589    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 01:05:11.639669 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:11.637839    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 01:05:11.639669 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:11.638020    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:05:16.455536 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:16.455500    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:05:16.459369 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:16.459345    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86dd9c8cf7-4rj2b"
Jul 14 01:05:22.637589 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:22.637554    2580 generic.go:356] "Generic (PLEG): container finished" podID="375faa1c-7017-4d2d-bcf4-b9db1a90c6b9" containerID="510b759ac40bbabaf1b4bcbb86d0ccf82264a55bf64bb4180617b6fd634faba3" exitCode=0
Jul 14 01:05:22.638071 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:22.637617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg" event={"ID":"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9","Type":"ContainerDied","Data":"510b759ac40bbabaf1b4bcbb86d0ccf82264a55bf64bb4180617b6fd634faba3"}
Jul 14 01:05:22.638071 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:22.637985    2580 scope.go:117] "RemoveContainer" containerID="510b759ac40bbabaf1b4bcbb86d0ccf82264a55bf64bb4180617b6fd634faba3"
Jul 14 01:05:23.643162 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:23.643131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zg4pg" event={"ID":"375faa1c-7017-4d2d-bcf4-b9db1a90c6b9","Type":"ContainerStarted","Data":"f338898a62d6d3d710d06bcf97914301fe1e786681be1fdf53eeead60d6855b0"}
Jul 14 01:05:24.637551 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:24.637492    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:05:35.641964 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:35.641910    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway; artifact err: get manifest: build image source: reading manifest sha256:6270079008ddaa493b09474324185a9224c3662bd93826c447f03f09cdd49a72 in registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9: received unexpected HTTP status: 502 Bad Gateway\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podUID="81cab238-19fe-4677-8643-e2ac58e6db64"
Jul 14 01:05:38.343500 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:38.343466    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:38.363140 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:38.363113    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:38.719038 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:38.718963    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:56.422080 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.422047    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:05:56.422792 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.422755    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="prometheus" containerID="cri-o://c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d" gracePeriod=600
Jul 14 01:05:56.422909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.422782    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy" containerID="cri-o://6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3" gracePeriod=600
Jul 14 01:05:56.422909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.422810    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="config-reloader" containerID="cri-o://a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248" gracePeriod=600
Jul 14 01:05:56.422909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.422821    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="thanos-sidecar" containerID="cri-o://57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8" gracePeriod=600
Jul 14 01:05:56.422909 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.422838    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27" gracePeriod=600
Jul 14 01:05:56.423094 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.422933    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy-web" containerID="cri-o://67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff" gracePeriod=600
Jul 14 01:05:56.771892 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.771867    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerID="f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27" exitCode=0
Jul 14 01:05:56.771892 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.771891    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerID="6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3" exitCode=0
Jul 14 01:05:56.772057 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.771898    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerID="57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8" exitCode=0
Jul 14 01:05:56.772057 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.771903    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerID="a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248" exitCode=0
Jul 14 01:05:56.772057 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.771909    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerID="c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d" exitCode=0
Jul 14 01:05:56.772057 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.771947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerDied","Data":"f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27"}
Jul 14 01:05:56.772057 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.772010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerDied","Data":"6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3"}
Jul 14 01:05:56.772057 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.772036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerDied","Data":"57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8"}
Jul 14 01:05:56.772234 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.772060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerDied","Data":"a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248"}
Jul 14 01:05:56.772234 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:56.772083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerDied","Data":"c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d"}
Jul 14 01:05:57.673286 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.673264    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.714770 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714746    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-kubelet-serving-ca-bundle\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.714914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-metrics-client-certs\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.714914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-config-out\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.714914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714828    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-thanos-prometheus-http-client-file\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.714914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-kube-rbac-proxy\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.714914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714876    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-1\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.714914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714899    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-trusted-ca-bundle\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714925    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-web-config\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714953    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.714980    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-0\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-db\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715035    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-tls-assets\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715094    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-grpc-tls\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715143    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-config\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9mwp\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-kube-api-access-h9mwp\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715212 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715200    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-tls\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715230    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-metrics-client-ca\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715263    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-2\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-serving-certs-ca-bundle\") pod \"ad9d622f-1143-4748-8a20-139e9095ccb5\" (UID: \"ad9d622f-1143-4748-8a20-139e9095ccb5\") "
Jul 14 01:05:57.715667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715429    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:05:57.715667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715471    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:05:57.715667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715660    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.716179 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.715681    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.716726 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.716256    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:05:57.717371 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.717088    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:05:57.720251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.718734    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 01:05:57.720251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.718812    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-kube-api-access-h9mwp" (OuterVolumeSpecName: "kube-api-access-h9mwp") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "kube-api-access-h9mwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:05:57.720251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.718951    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:05:57.720251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.719205    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:05:57.720251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.719317    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.720251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.720118    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:05:57.720251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.720206    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-config" (OuterVolumeSpecName: "config") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.720831 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.720397    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.720831 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.720485    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.720831 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.720772    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.721073 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.721047    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.721146 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.721114    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-config-out" (OuterVolumeSpecName: "config-out") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 01:05:57.721543 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.721497    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:05:57.721685 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.721665    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.722256 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.722240    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.733408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.733383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-web-config" (OuterVolumeSpecName: "web-config") pod "ad9d622f-1143-4748-8a20-139e9095ccb5" (UID: "ad9d622f-1143-4748-8a20-139e9095ccb5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:05:57.783107 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.783076    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerID="67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff" exitCode=0
Jul 14 01:05:57.783235 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.783144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerDied","Data":"67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff"}
Jul 14 01:05:57.783235 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.783173    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.783235 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.783231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ad9d622f-1143-4748-8a20-139e9095ccb5","Type":"ContainerDied","Data":"dfc1f8ccb40116ce82d0f468657d3fad96d5ddc85cd3f77eb4cadb8870b3949c"}
Jul 14 01:05:57.783331 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.783265    2580 scope.go:117] "RemoveContainer" containerID="f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27"
Jul 14 01:05:57.790658 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.790638    2580 scope.go:117] "RemoveContainer" containerID="6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3"
Jul 14 01:05:57.797831 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.797814    2580 scope.go:117] "RemoveContainer" containerID="67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff"
Jul 14 01:05:57.804856 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.804835    2580 scope.go:117] "RemoveContainer" containerID="57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8"
Jul 14 01:05:57.805474 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.805452    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:05:57.809735 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.809716    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:05:57.812054 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.812040    2580 scope.go:117] "RemoveContainer" containerID="a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248"
Jul 14 01:05:57.816422 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816404    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-grpc-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816425    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-config\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816438    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9mwp\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-kube-api-access-h9mwp\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816452    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816466    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-metrics-client-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816478    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816492    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816507    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816548 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816538    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-metrics-client-certs\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816551    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-config-out\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816564    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816577    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-kube-rbac-proxy\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816591    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816628    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-web-config\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816643    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad9d622f-1143-4748-8a20-139e9095ccb5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816657    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816671    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ad9d622f-1143-4748-8a20-139e9095ccb5-prometheus-k8s-db\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.816834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.816685    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad9d622f-1143-4748-8a20-139e9095ccb5-tls-assets\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:05:57.818652 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.818634    2580 scope.go:117] "RemoveContainer" containerID="c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d"
Jul 14 01:05:57.825714 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.825698    2580 scope.go:117] "RemoveContainer" containerID="3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488"
Jul 14 01:05:57.832614 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.832588    2580 scope.go:117] "RemoveContainer" containerID="f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27"
Jul 14 01:05:57.832890 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:57.832871    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27\": container with ID starting with f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27 not found: ID does not exist" containerID="f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27"
Jul 14 01:05:57.832983 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.832931    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27"} err="failed to get container status \"f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27\": rpc error: code = NotFound desc = could not find container \"f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27\": container with ID starting with f45b9b2c8d8bce041719eb6c8490634ecd0dddf77f51667e2da082742463bc27 not found: ID does not exist"
Jul 14 01:05:57.832983 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.832973    2580 scope.go:117] "RemoveContainer" containerID="6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3"
Jul 14 01:05:57.833225 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:57.833209    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3\": container with ID starting with 6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3 not found: ID does not exist" containerID="6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3"
Jul 14 01:05:57.833276 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833231    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3"} err="failed to get container status \"6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3\": rpc error: code = NotFound desc = could not find container \"6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3\": container with ID starting with 6c4b0ef688208feaf2989f52fbb88251332d64bf7c894da90b64e18cea7da2a3 not found: ID does not exist"
Jul 14 01:05:57.833276 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833249    2580 scope.go:117] "RemoveContainer" containerID="67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff"
Jul 14 01:05:57.833461 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833443    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:05:57.833640 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:57.833503    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff\": container with ID starting with 67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff not found: ID does not exist" containerID="67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff"
Jul 14 01:05:57.833684 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833660    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff"} err="failed to get container status \"67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff\": rpc error: code = NotFound desc = could not find container \"67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff\": container with ID starting with 67d428d3d79b22bd79e6148d5033c186a56eca5ecd70e62d2936c6693548fbff not found: ID does not exist"
Jul 14 01:05:57.833720 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833688    2580 scope.go:117] "RemoveContainer" containerID="57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8"
Jul 14 01:05:57.833751 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833737    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="init-config-reloader"
Jul 14 01:05:57.833751 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833748    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="init-config-reloader"
Jul 14 01:05:57.833814 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833758    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy"
Jul 14 01:05:57.833814 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833767    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy"
Jul 14 01:05:57.833814 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833778    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="thanos-sidecar"
Jul 14 01:05:57.833814 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833786    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="thanos-sidecar"
Jul 14 01:05:57.833814 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833801    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f305f6e3-06c6-4680-a58e-51b371866bea" containerName="registry"
Jul 14 01:05:57.833814 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833809    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f305f6e3-06c6-4680-a58e-51b371866bea" containerName="registry"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833818    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy-thanos"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833827    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy-thanos"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833837    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="prometheus"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833845    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="prometheus"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833854    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="config-reloader"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833863    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="config-reloader"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833874    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy-web"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833882    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy-web"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833958    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy-web"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833968    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="prometheus"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833976    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="thanos-sidecar"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:57.833971    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8\": container with ID starting with 57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8 not found: ID does not exist" containerID="57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8"
Jul 14 01:05:57.834009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833997    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8"} err="failed to get container status \"57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8\": rpc error: code = NotFound desc = could not find container \"57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8\": container with ID starting with 57f2690d83c6dc78325915fc329633f1e5dd3ba6bd96e233489d39e622f853a8 not found: ID does not exist"
Jul 14 01:05:57.834403 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834018    2580 scope.go:117] "RemoveContainer" containerID="a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248"
Jul 14 01:05:57.834403 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.833981    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy-thanos"
Jul 14 01:05:57.834403 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834056    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="kube-rbac-proxy"
Jul 14 01:05:57.834403 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834064    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f305f6e3-06c6-4680-a58e-51b371866bea" containerName="registry"
Jul 14 01:05:57.834403 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834071    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" containerName="config-reloader"
Jul 14 01:05:57.834403 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:57.834214    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248\": container with ID starting with a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248 not found: ID does not exist" containerID="a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248"
Jul 14 01:05:57.834403 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834229    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248"} err="failed to get container status \"a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248\": rpc error: code = NotFound desc = could not find container \"a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248\": container with ID starting with a0235d2be29c8f0118de3a3f1262d061b044d0590181eecf972f3c3c18a0a248 not found: ID does not exist"
Jul 14 01:05:57.834403 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834242    2580 scope.go:117] "RemoveContainer" containerID="c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d"
Jul 14 01:05:57.834704 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:57.834454    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d\": container with ID starting with c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d not found: ID does not exist" containerID="c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d"
Jul 14 01:05:57.834704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834479    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d"} err="failed to get container status \"c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d\": rpc error: code = NotFound desc = could not find container \"c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d\": container with ID starting with c29596d60678223f8a86a2ece432e88fb7f963d3c9e767fba9ab989bfd054e7d not found: ID does not exist"
Jul 14 01:05:57.834704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834490    2580 scope.go:117] "RemoveContainer" containerID="3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488"
Jul 14 01:05:57.834805 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:05:57.834744    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488\": container with ID starting with 3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488 not found: ID does not exist" containerID="3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488"
Jul 14 01:05:57.834805 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.834760    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488"} err="failed to get container status \"3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488\": rpc error: code = NotFound desc = could not find container \"3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488\": container with ID starting with 3be8f24df3d3b2e61d4954ec7c372fe424f0335bcbc42cfb3fe5ca5cc06da488 not found: ID does not exist"
Jul 14 01:05:57.842596 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.842578    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.844395 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.844376    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 01:05:57.844485 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.844457    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 01:05:57.844585 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.844502    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 01:05:57.844585 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.844459    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 01:05:57.844699 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.844659    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 01:05:57.844829 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.844814    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 01:05:57.844985 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.844964    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 01:05:57.845055 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.844987    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jjvpc\""
Jul 14 01:05:57.845153 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.845134    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 01:05:57.845259 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.845239    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fv8oalj5f9kq0\""
Jul 14 01:05:57.845259 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.845246    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 01:05:57.845374 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.845314    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 01:05:57.845416 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.845392    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 01:05:57.845500 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.845487    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 01:05:57.847713 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.847697    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 01:05:57.851815 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.851793    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 01:05:57.851921 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.851846    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:05:57.917859 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.917827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.917859 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.917862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.917998 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.917880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-config\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.917998 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.917896    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.917998 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.917968    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a96771c-e57e-4b42-9b79-e5f6a1adc398-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.917998 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.917996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918118 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqqtx\" (UniqueName: \"kubernetes.io/projected/8a96771c-e57e-4b42-9b79-e5f6a1adc398-kube-api-access-zqqtx\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918118 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a96771c-e57e-4b42-9b79-e5f6a1adc398-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918118 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918118 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918118 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918257 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918257 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918257 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918257 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918257 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918396 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918396 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918396 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:57.918396 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:57.918376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019498 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-config\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019800 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a96771c-e57e-4b42-9b79-e5f6a1adc398-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019800 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019800 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqqtx\" (UniqueName: \"kubernetes.io/projected/8a96771c-e57e-4b42-9b79-e5f6a1adc398-kube-api-access-zqqtx\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a96771c-e57e-4b42-9b79-e5f6a1adc398-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.019956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.020248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.019983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.020248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.020009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.020248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.020055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.020248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.020080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.020248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.020100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.020248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.020138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.020728 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.020264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.020728 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.020515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.021550 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.021499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.022556 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.022510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.022818 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.022753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a96771c-e57e-4b42-9b79-e5f6a1adc398-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.023690 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.023662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.023802 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.023776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.023898 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.023799    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.024828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.024198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.024828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.024201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.024828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.024328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.024828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.024488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8a96771c-e57e-4b42-9b79-e5f6a1adc398-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.024828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.024773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.024828 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.024789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a96771c-e57e-4b42-9b79-e5f6a1adc398-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.025160 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.024873    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-config\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.025275 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.025257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.025545 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.025501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.025619 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.025586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.026056 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.026039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a96771c-e57e-4b42-9b79-e5f6a1adc398-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.027448 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.027432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqqtx\" (UniqueName: \"kubernetes.io/projected/8a96771c-e57e-4b42-9b79-e5f6a1adc398-kube-api-access-zqqtx\") pod \"prometheus-k8s-0\" (UID: \"8a96771c-e57e-4b42-9b79-e5f6a1adc398\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.153678 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.153660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:05:58.290098 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.290032    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 01:05:58.293454 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:05:58.293427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a96771c_e57e_4b42_9b79_e5f6a1adc398.slice/crio-05ec4d7ce1b183d85306183bad71abda04fb57c7ee683b6b03fbdbdacc433baa WatchSource:0}: Error finding container 05ec4d7ce1b183d85306183bad71abda04fb57c7ee683b6b03fbdbdacc433baa: Status 404 returned error can't find the container with id 05ec4d7ce1b183d85306183bad71abda04fb57c7ee683b6b03fbdbdacc433baa
Jul 14 01:05:58.787641 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.787613    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a96771c-e57e-4b42-9b79-e5f6a1adc398" containerID="5af765f4c08bc96ddddf32743059400ed4da8595415cc7eb6b829fd0973d7462" exitCode=0
Jul 14 01:05:58.788015 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.787702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a96771c-e57e-4b42-9b79-e5f6a1adc398","Type":"ContainerDied","Data":"5af765f4c08bc96ddddf32743059400ed4da8595415cc7eb6b829fd0973d7462"}
Jul 14 01:05:58.788015 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:58.787787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a96771c-e57e-4b42-9b79-e5f6a1adc398","Type":"ContainerStarted","Data":"05ec4d7ce1b183d85306183bad71abda04fb57c7ee683b6b03fbdbdacc433baa"}
Jul 14 01:05:59.646270 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:59.646176    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad9d622f-1143-4748-8a20-139e9095ccb5" path="/var/lib/kubelet/pods/ad9d622f-1143-4748-8a20-139e9095ccb5/volumes"
Jul 14 01:05:59.795730 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:59.795692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a96771c-e57e-4b42-9b79-e5f6a1adc398","Type":"ContainerStarted","Data":"f92dfbb45f8dae37c37ff5f36f899a3bd680ece75a7f8f99251ba28d503a5c08"}
Jul 14 01:05:59.796086 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:59.795761    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a96771c-e57e-4b42-9b79-e5f6a1adc398","Type":"ContainerStarted","Data":"479b382557448190e33ee3c8ae892b44105dc0eae24ccb5fbf0062899efc0841"}
Jul 14 01:05:59.796086 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:59.795801    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a96771c-e57e-4b42-9b79-e5f6a1adc398","Type":"ContainerStarted","Data":"239cbfebc9113487f3781110f24d3654ec11239abe8fcb0aef1518a3fc877e17"}
Jul 14 01:05:59.796086 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:59.795829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a96771c-e57e-4b42-9b79-e5f6a1adc398","Type":"ContainerStarted","Data":"0c32907efb32e4a40ca80852534de4a7a09c23fff445328d65afd3c1b5c64ae9"}
Jul 14 01:05:59.796086 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:59.795854    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a96771c-e57e-4b42-9b79-e5f6a1adc398","Type":"ContainerStarted","Data":"b4fe507be4a155fbe0814c36569d2f58c3621e7190f7330d525c11e067182095"}
Jul 14 01:05:59.796086 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:59.795895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a96771c-e57e-4b42-9b79-e5f6a1adc398","Type":"ContainerStarted","Data":"d846c45f6d6ff89186cd144647b5a783fb977f33ca84d5d35b076020b2c61007"}
Jul 14 01:05:59.822322 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:05:59.821870    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.80165927 podStartE2EDuration="2.821850532s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.191262ms" isStatefulPod=false podCreationTimestamp="2026-07-14 01:05:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:05:59.819801307 +0000 UTC m=+350.714651946" watchObservedRunningTime="2026-07-14 01:05:59.821850532 +0000 UTC m=+350.716701173"
Jul 14 01:06:03.154405 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:06:03.154374    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:06:18.871071 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:06:18.871033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" event={"ID":"81cab238-19fe-4677-8643-e2ac58e6db64","Type":"ContainerStarted","Data":"e4874f52b45067d8a68000af526aee575439416e6f719a97ee66ebb60a3dda8b"}
Jul 14 01:06:18.887839 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:06:18.887776    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74785d7fdd-c5m4g" podStartSLOduration=1.237306781 podStartE2EDuration="4m16.887759861s" totalImagesPullingTime="4m15.65045308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:02:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=4 observedRunningTime="2026-07-14 01:06:18.887123464 +0000 UTC m=+369.781974102" watchObservedRunningTime="2026-07-14 01:06:18.887759861 +0000 UTC m=+369.782610502"
Jul 14 01:06:58.154899 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:06:58.154870    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:06:58.170436 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:06:58.170414    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:06:59.031589 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:06:59.031563    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 01:07:44.018926 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.018848    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"]
Jul 14 01:07:44.022048 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.022027    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"
Jul 14 01:07:44.028456 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.028429    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 14 01:07:44.028990 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.028973    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 14 01:07:44.029245 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.029062    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-646tq\""
Jul 14 01:07:44.032177 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.032156    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"]
Jul 14 01:07:44.105053 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.105029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8lxp\" (UniqueName: \"kubernetes.io/projected/0b316b17-7a8e-40b4-bec6-6945c5fe4fe7-kube-api-access-m8lxp\") pod \"openshift-lws-operator-fd8ccff4c-wdcg7\" (UID: \"0b316b17-7a8e-40b4-bec6-6945c5fe4fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"
Jul 14 01:07:44.105165 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.105093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b316b17-7a8e-40b4-bec6-6945c5fe4fe7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wdcg7\" (UID: \"0b316b17-7a8e-40b4-bec6-6945c5fe4fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"
Jul 14 01:07:44.206160 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.206129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b316b17-7a8e-40b4-bec6-6945c5fe4fe7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wdcg7\" (UID: \"0b316b17-7a8e-40b4-bec6-6945c5fe4fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"
Jul 14 01:07:44.206278 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.206200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8lxp\" (UniqueName: \"kubernetes.io/projected/0b316b17-7a8e-40b4-bec6-6945c5fe4fe7-kube-api-access-m8lxp\") pod \"openshift-lws-operator-fd8ccff4c-wdcg7\" (UID: \"0b316b17-7a8e-40b4-bec6-6945c5fe4fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"
Jul 14 01:07:44.206503 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.206483    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b316b17-7a8e-40b4-bec6-6945c5fe4fe7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wdcg7\" (UID: \"0b316b17-7a8e-40b4-bec6-6945c5fe4fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"
Jul 14 01:07:44.212867 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.212847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8lxp\" (UniqueName: \"kubernetes.io/projected/0b316b17-7a8e-40b4-bec6-6945c5fe4fe7-kube-api-access-m8lxp\") pod \"openshift-lws-operator-fd8ccff4c-wdcg7\" (UID: \"0b316b17-7a8e-40b4-bec6-6945c5fe4fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"
Jul 14 01:07:44.338325 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.338271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"
Jul 14 01:07:44.465443 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:44.465414    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7"]
Jul 14 01:07:44.468771 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:07:44.468746    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b316b17_7a8e_40b4_bec6_6945c5fe4fe7.slice/crio-906105eeec65de8fe91bb2b584fdc6de8187d64da0374a0ea67fe1c0f770451f WatchSource:0}: Error finding container 906105eeec65de8fe91bb2b584fdc6de8187d64da0374a0ea67fe1c0f770451f: Status 404 returned error can't find the container with id 906105eeec65de8fe91bb2b584fdc6de8187d64da0374a0ea67fe1c0f770451f
Jul 14 01:07:45.197742 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:45.197713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7" event={"ID":"0b316b17-7a8e-40b4-bec6-6945c5fe4fe7","Type":"ContainerStarted","Data":"906105eeec65de8fe91bb2b584fdc6de8187d64da0374a0ea67fe1c0f770451f"}
Jul 14 01:07:49.216863 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:49.216784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7" event={"ID":"0b316b17-7a8e-40b4-bec6-6945c5fe4fe7","Type":"ContainerStarted","Data":"5c5a1642358f1719cd1250f4876dbb3af6be69c8988ba78cffee0a824c24da04"}
Jul 14 01:07:49.231576 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:07:49.231509    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wdcg7" podStartSLOduration=1.368754429 podStartE2EDuration="5.231494965s" totalImagesPullingTime="3.862740536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:07:49.229646402 +0000 UTC m=+460.124497041" watchObservedRunningTime="2026-07-14 01:07:49.231494965 +0000 UTC m=+460.126345604"
Jul 14 01:08:02.368234 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.368203    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"]
Jul 14 01:08:02.371360 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.371340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.373353 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.373330    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 14 01:08:02.373737 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.373712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 14 01:08:02.373737 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.373733    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-92tqw\""
Jul 14 01:08:02.373913 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.373800    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 14 01:08:02.378503 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.378477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"]
Jul 14 01:08:02.436317 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.436292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5sv\" (UniqueName: \"kubernetes.io/projected/89089398-cf9e-4ffd-931e-7e45bf12f4bc-kube-api-access-cn5sv\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.436465 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.436324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/89089398-cf9e-4ffd-931e-7e45bf12f4bc-manager-config\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.436465 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.436363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89089398-cf9e-4ffd-931e-7e45bf12f4bc-cert\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.436465 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.436380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/89089398-cf9e-4ffd-931e-7e45bf12f4bc-metrics-cert\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.537880 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.537845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5sv\" (UniqueName: \"kubernetes.io/projected/89089398-cf9e-4ffd-931e-7e45bf12f4bc-kube-api-access-cn5sv\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.538064 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.537885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/89089398-cf9e-4ffd-931e-7e45bf12f4bc-manager-config\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.538064 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.537932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89089398-cf9e-4ffd-931e-7e45bf12f4bc-cert\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.538181 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.538088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/89089398-cf9e-4ffd-931e-7e45bf12f4bc-metrics-cert\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.538680 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.538651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/89089398-cf9e-4ffd-931e-7e45bf12f4bc-manager-config\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.540538 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.540503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89089398-cf9e-4ffd-931e-7e45bf12f4bc-cert\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.540659 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.540594    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/89089398-cf9e-4ffd-931e-7e45bf12f4bc-metrics-cert\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.546123 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.546092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5sv\" (UniqueName: \"kubernetes.io/projected/89089398-cf9e-4ffd-931e-7e45bf12f4bc-kube-api-access-cn5sv\") pod \"lws-controller-manager-78f77bcdb4-246v4\" (UID: \"89089398-cf9e-4ffd-931e-7e45bf12f4bc\") " pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.681554 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.681470    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:02.819613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:02.819592    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"]
Jul 14 01:08:02.822085 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:08:02.822051    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89089398_cf9e_4ffd_931e_7e45bf12f4bc.slice/crio-f453da61817284df69de24b190d300e76c01a679a9db6f87b1bef6fb396ecdca WatchSource:0}: Error finding container f453da61817284df69de24b190d300e76c01a679a9db6f87b1bef6fb396ecdca: Status 404 returned error can't find the container with id f453da61817284df69de24b190d300e76c01a679a9db6f87b1bef6fb396ecdca
Jul 14 01:08:03.269890 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:03.269855    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4" event={"ID":"89089398-cf9e-4ffd-931e-7e45bf12f4bc","Type":"ContainerStarted","Data":"f453da61817284df69de24b190d300e76c01a679a9db6f87b1bef6fb396ecdca"}
Jul 14 01:08:05.280002 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:05.279969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4" event={"ID":"89089398-cf9e-4ffd-931e-7e45bf12f4bc","Type":"ContainerStarted","Data":"293b20c848c464e9b9fc9704978733f950c141927b2b738d32f6e8c295a39994"}
Jul 14 01:08:05.280357 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:05.280041    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:08.684817 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.684765    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4" podStartSLOduration=4.70561625 podStartE2EDuration="6.684748403s" totalImagesPullingTime="1.979132153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:08:05.293215633 +0000 UTC m=+476.188066273" watchObservedRunningTime="2026-07-14 01:08:08.684748403 +0000 UTC m=+479.579599049"
Jul 14 01:08:08.685180 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.685040    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"]
Jul 14 01:08:08.691441 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.691426    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.693443 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.693416    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 01:08:08.693583 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.693503    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 01:08:08.693660 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.693605    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 01:08:08.693660 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.693605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fbcld\""
Jul 14 01:08:08.693866 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.693850    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 01:08:08.704473 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.704448    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"]
Jul 14 01:08:08.798195 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.798167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-tmp\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.798357 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.798205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.798357 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.798312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.798357 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.798354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blfsl\" (UniqueName: \"kubernetes.io/projected/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-kube-api-access-blfsl\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.798732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.798453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-manifests\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.899226 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.899197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-manifests\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.899344 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.899244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-tmp\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.899344 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.899265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.899344 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.899314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.899480 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.899341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blfsl\" (UniqueName: \"kubernetes.io/projected/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-kube-api-access-blfsl\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.899701 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.899667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-manifests\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.901668 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.901643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-tmp\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.901752 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.901722    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.901794 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.901761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:08.907960 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:08.907937    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blfsl\" (UniqueName: \"kubernetes.io/projected/a19ea936-a202-4f6b-8d8d-95d5c1a2bab7-kube-api-access-blfsl\") pod \"opendatahub-operator-controller-manager-bc876bb4c-jc6xk\" (UID: \"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7\") " pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:09.003224 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:09.003201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:09.145917 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:09.145822    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"]
Jul 14 01:08:09.148947 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:08:09.148920    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda19ea936_a202_4f6b_8d8d_95d5c1a2bab7.slice/crio-d1e32ba7f94af959fe61090fba16f56e2111baba06e4a3842ed02ea66f7fdbf8 WatchSource:0}: Error finding container d1e32ba7f94af959fe61090fba16f56e2111baba06e4a3842ed02ea66f7fdbf8: Status 404 returned error can't find the container with id d1e32ba7f94af959fe61090fba16f56e2111baba06e4a3842ed02ea66f7fdbf8
Jul 14 01:08:09.295571 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:09.295478    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk" event={"ID":"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7","Type":"ContainerStarted","Data":"d1e32ba7f94af959fe61090fba16f56e2111baba06e4a3842ed02ea66f7fdbf8"}
Jul 14 01:08:13.316774 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:13.316739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk" event={"ID":"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7","Type":"ContainerStarted","Data":"7342b857aa70092e81536a4b826ec40f543c1a748aced03c32c5d43af6952b43"}
Jul 14 01:08:14.323122 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:14.323091    2580 generic.go:356] "Generic (PLEG): container finished" podID="a19ea936-a202-4f6b-8d8d-95d5c1a2bab7" containerID="7342b857aa70092e81536a4b826ec40f543c1a748aced03c32c5d43af6952b43" exitCode=0
Jul 14 01:08:14.323481 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:14.323170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk" event={"ID":"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7","Type":"ContainerDied","Data":"7342b857aa70092e81536a4b826ec40f543c1a748aced03c32c5d43af6952b43"}
Jul 14 01:08:15.330300 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:15.330259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk" event={"ID":"a19ea936-a202-4f6b-8d8d-95d5c1a2bab7","Type":"ContainerStarted","Data":"adce80977a624c4498eedf58755a92119d8dee95b27f17c31615526ab31c54d6"}
Jul 14 01:08:15.330704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:15.330339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:15.350595 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:15.350500    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk" podStartSLOduration=2.337294535 podStartE2EDuration="7.350458818s" totalImagesPullingTime="3.976222893s" totalInitContainerRuntime="1.03694139s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:08:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:08:15.349653906 +0000 UTC m=+486.244504547" watchObservedRunningTime="2026-07-14 01:08:15.350458818 +0000 UTC m=+486.245309462"
Jul 14 01:08:16.285813 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:16.285782    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78f77bcdb4-246v4"
Jul 14 01:08:26.337424 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:26.337395    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bc876bb4c-jc6xk"
Jul 14 01:08:29.190984 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.190950    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"]
Jul 14 01:08:29.200254 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.200229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.205772 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.205745    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 01:08:29.206722 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.206032    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-64cwf\""
Jul 14 01:08:29.206722 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.206219    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 01:08:29.206722 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.206375    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 01:08:29.206722 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.206374    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 01:08:29.211225 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.211203    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"]
Jul 14 01:08:29.359160 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.359126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzlfp\" (UniqueName: \"kubernetes.io/projected/72ca077f-314f-4b4d-80ba-d9f7f89fd270-kube-api-access-bzlfp\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.359351 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.359176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72ca077f-314f-4b4d-80ba-d9f7f89fd270-tls-certs\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.359351 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.359252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ca077f-314f-4b4d-80ba-d9f7f89fd270-tmp\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.459854 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.459758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzlfp\" (UniqueName: \"kubernetes.io/projected/72ca077f-314f-4b4d-80ba-d9f7f89fd270-kube-api-access-bzlfp\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.459854 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.459829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72ca077f-314f-4b4d-80ba-d9f7f89fd270-tls-certs\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.460078 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.459904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ca077f-314f-4b4d-80ba-d9f7f89fd270-tmp\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.462555 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.462508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ca077f-314f-4b4d-80ba-d9f7f89fd270-tmp\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.462888 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.462869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72ca077f-314f-4b4d-80ba-d9f7f89fd270-tls-certs\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.472069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.472040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzlfp\" (UniqueName: \"kubernetes.io/projected/72ca077f-314f-4b4d-80ba-d9f7f89fd270-kube-api-access-bzlfp\") pod \"kube-auth-proxy-6d697d898-xdr7l\" (UID: \"72ca077f-314f-4b4d-80ba-d9f7f89fd270\") " pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.512580 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.512542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"
Jul 14 01:08:29.681350 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:29.681330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d697d898-xdr7l"]
Jul 14 01:08:29.683580 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:08:29.683554    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ca077f_314f_4b4d_80ba_d9f7f89fd270.slice/crio-67efa765c57542f17bd7de9d73606e1c2c667f40d4e4832a17c0c2613572089e WatchSource:0}: Error finding container 67efa765c57542f17bd7de9d73606e1c2c667f40d4e4832a17c0c2613572089e: Status 404 returned error can't find the container with id 67efa765c57542f17bd7de9d73606e1c2c667f40d4e4832a17c0c2613572089e
Jul 14 01:08:30.402668 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:30.402630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l" event={"ID":"72ca077f-314f-4b4d-80ba-d9f7f89fd270","Type":"ContainerStarted","Data":"67efa765c57542f17bd7de9d73606e1c2c667f40d4e4832a17c0c2613572089e"}
Jul 14 01:08:33.420464 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:33.420369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l" event={"ID":"72ca077f-314f-4b4d-80ba-d9f7f89fd270","Type":"ContainerStarted","Data":"32cf5af5c4c3a0f6a59f3c1bcc8138e9f3cf7fb732c8b17877b1bad251667909"}
Jul 14 01:08:33.435505 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:08:33.435449    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6d697d898-xdr7l" podStartSLOduration=1.017976296 podStartE2EDuration="4.435428805s" totalImagesPullingTime="3.417452509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:08:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:08:33.434710699 +0000 UTC m=+504.329561338" watchObservedRunningTime="2026-07-14 01:08:33.435428805 +0000 UTC m=+504.330279447"
Jul 14 01:10:08.063761 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.063654    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"]
Jul 14 01:10:08.074152 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.069031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.085653 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.085585    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 14 01:10:08.086133 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.086084    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9l8cc\""
Jul 14 01:10:08.086290 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.086250    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 01:10:08.086650 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.086612    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 14 01:10:08.087053 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.086679    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 01:10:08.091987 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.091918    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"]
Jul 14 01:10:08.159293 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.159217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa784b3b-bd5f-4714-8877-816544fdfdaf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.159293 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.159314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb9zp\" (UniqueName: \"kubernetes.io/projected/fa784b3b-bd5f-4714-8877-816544fdfdaf-kube-api-access-cb9zp\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.159830 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.159480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fa784b3b-bd5f-4714-8877-816544fdfdaf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.260325 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.260287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fa784b3b-bd5f-4714-8877-816544fdfdaf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.260566 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.260378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa784b3b-bd5f-4714-8877-816544fdfdaf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.260566 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.260422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb9zp\" (UniqueName: \"kubernetes.io/projected/fa784b3b-bd5f-4714-8877-816544fdfdaf-kube-api-access-cb9zp\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.260566 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:10:08.260550    2580 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 14 01:10:08.260752 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:10:08.260629    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa784b3b-bd5f-4714-8877-816544fdfdaf-plugin-serving-cert podName:fa784b3b-bd5f-4714-8877-816544fdfdaf nodeName:}" failed. No retries permitted until 2026-07-14 01:10:08.760609645 +0000 UTC m=+599.655460265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/fa784b3b-bd5f-4714-8877-816544fdfdaf-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-frt9g" (UID: "fa784b3b-bd5f-4714-8877-816544fdfdaf") : secret "plugin-serving-cert" not found
Jul 14 01:10:08.261083 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.261058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fa784b3b-bd5f-4714-8877-816544fdfdaf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.271159 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.271133    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb9zp\" (UniqueName: \"kubernetes.io/projected/fa784b3b-bd5f-4714-8877-816544fdfdaf-kube-api-access-cb9zp\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.765315 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.765199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa784b3b-bd5f-4714-8877-816544fdfdaf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:08.767757 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:08.767733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa784b3b-bd5f-4714-8877-816544fdfdaf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-frt9g\" (UID: \"fa784b3b-bd5f-4714-8877-816544fdfdaf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:09.004223 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:09.004182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"
Jul 14 01:10:09.142264 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:09.142232    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g"]
Jul 14 01:10:09.145588 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:10:09.145560    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa784b3b_bd5f_4714_8877_816544fdfdaf.slice/crio-2760df4dabae359e468d937a5794d590fc4b55b123952c723cac85823bed2024 WatchSource:0}: Error finding container 2760df4dabae359e468d937a5794d590fc4b55b123952c723cac85823bed2024: Status 404 returned error can't find the container with id 2760df4dabae359e468d937a5794d590fc4b55b123952c723cac85823bed2024
Jul 14 01:10:09.852009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:09.851967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g" event={"ID":"fa784b3b-bd5f-4714-8877-816544fdfdaf","Type":"ContainerStarted","Data":"2760df4dabae359e468d937a5794d590fc4b55b123952c723cac85823bed2024"}
Jul 14 01:10:33.980312 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:33.980276    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g" event={"ID":"fa784b3b-bd5f-4714-8877-816544fdfdaf","Type":"ContainerStarted","Data":"920b6fed837255a02f6e5020c53a8feb4ec7377c55b37e6c61084ed57a4410ac"}
Jul 14 01:10:33.995761 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:33.995713    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-frt9g" podStartSLOduration=2.159206729 podStartE2EDuration="25.995696415s" totalImagesPullingTime="23.836489686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:10:33.99383825 +0000 UTC m=+624.888688883" watchObservedRunningTime="2026-07-14 01:10:33.995696415 +0000 UTC m=+624.890547054"
Jul 14 01:10:52.739878 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.739836    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"]
Jul 14 01:10:52.742449 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.742425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:52.744333 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.744304    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 14 01:10:52.752993 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.752966    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"]
Jul 14 01:10:52.782471 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.782439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tmlb\" (UniqueName: \"kubernetes.io/projected/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-kube-api-access-5tmlb\") pod \"limitador-limitador-64c94fd64f-8xj2b\" (UID: \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:52.782637 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.782486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-config-file\") pod \"limitador-limitador-64c94fd64f-8xj2b\" (UID: \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:52.793667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.793640    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"]
Jul 14 01:10:52.883061 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.883028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-config-file\") pod \"limitador-limitador-64c94fd64f-8xj2b\" (UID: \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:52.883222 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.883102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tmlb\" (UniqueName: \"kubernetes.io/projected/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-kube-api-access-5tmlb\") pod \"limitador-limitador-64c94fd64f-8xj2b\" (UID: \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:52.883697 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.883678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-config-file\") pod \"limitador-limitador-64c94fd64f-8xj2b\" (UID: \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:52.893125 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:52.893104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tmlb\" (UniqueName: \"kubernetes.io/projected/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-kube-api-access-5tmlb\") pod \"limitador-limitador-64c94fd64f-8xj2b\" (UID: \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:53.054447 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.054409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:53.185499 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.185474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"]
Jul 14 01:10:53.189223 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:10:53.189194    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ed26e5_d3b1_4b52_b87c_fa59b0287959.slice/crio-ab398bd4600ccb659543d9b10fbb3bbca013c33bd2ccca849c2a100bba5b04ee WatchSource:0}: Error finding container ab398bd4600ccb659543d9b10fbb3bbca013c33bd2ccca849c2a100bba5b04ee: Status 404 returned error can't find the container with id ab398bd4600ccb659543d9b10fbb3bbca013c33bd2ccca849c2a100bba5b04ee
Jul 14 01:10:53.191351 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.191334    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 01:10:53.572735 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.572700    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-s92qp"]
Jul 14 01:10:53.579018 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.578978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-s92qp"
Jul 14 01:10:53.581342 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.581304    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2v97x\""
Jul 14 01:10:53.582819 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.582793    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s92qp"]
Jul 14 01:10:53.690048 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.690017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwbpl\" (UniqueName: \"kubernetes.io/projected/424265bf-371a-4549-a93f-769b82de00b4-kube-api-access-pwbpl\") pod \"authorino-75d556bd77-s92qp\" (UID: \"424265bf-371a-4549-a93f-769b82de00b4\") " pod="kuadrant-system/authorino-75d556bd77-s92qp"
Jul 14 01:10:53.791309 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.791267    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwbpl\" (UniqueName: \"kubernetes.io/projected/424265bf-371a-4549-a93f-769b82de00b4-kube-api-access-pwbpl\") pod \"authorino-75d556bd77-s92qp\" (UID: \"424265bf-371a-4549-a93f-769b82de00b4\") " pod="kuadrant-system/authorino-75d556bd77-s92qp"
Jul 14 01:10:53.798227 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.798201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwbpl\" (UniqueName: \"kubernetes.io/projected/424265bf-371a-4549-a93f-769b82de00b4-kube-api-access-pwbpl\") pod \"authorino-75d556bd77-s92qp\" (UID: \"424265bf-371a-4549-a93f-769b82de00b4\") " pod="kuadrant-system/authorino-75d556bd77-s92qp"
Jul 14 01:10:53.890431 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:53.890356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-s92qp"
Jul 14 01:10:54.075371 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:54.073627    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s92qp"]
Jul 14 01:10:54.078843 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:10:54.078643    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod424265bf_371a_4549_a93f_769b82de00b4.slice/crio-cdd194857b2964d1c95b7b3424a9851098361816300403628155768ed1e97014 WatchSource:0}: Error finding container cdd194857b2964d1c95b7b3424a9851098361816300403628155768ed1e97014: Status 404 returned error can't find the container with id cdd194857b2964d1c95b7b3424a9851098361816300403628155768ed1e97014
Jul 14 01:10:54.081971 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:54.081933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b" event={"ID":"f5ed26e5-d3b1-4b52-b87c-fa59b0287959","Type":"ContainerStarted","Data":"ab398bd4600ccb659543d9b10fbb3bbca013c33bd2ccca849c2a100bba5b04ee"}
Jul 14 01:10:55.091065 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:55.091024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-s92qp" event={"ID":"424265bf-371a-4549-a93f-769b82de00b4","Type":"ContainerStarted","Data":"cdd194857b2964d1c95b7b3424a9851098361816300403628155768ed1e97014"}
Jul 14 01:10:58.107758 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:58.107669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b" event={"ID":"f5ed26e5-d3b1-4b52-b87c-fa59b0287959","Type":"ContainerStarted","Data":"1c6f916257f7e5d9d57df85899937444eb851bf211705f4d956a7feea751df2a"}
Jul 14 01:10:58.108208 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:58.107794    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:10:58.109349 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:58.109328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-s92qp" event={"ID":"424265bf-371a-4549-a93f-769b82de00b4","Type":"ContainerStarted","Data":"36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30"}
Jul 14 01:10:58.121533 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:58.121481    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b" podStartSLOduration=1.545615358 podStartE2EDuration="6.121464166s" totalImagesPullingTime="4.575848808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:10:58.119964722 +0000 UTC m=+649.014815361" watchObservedRunningTime="2026-07-14 01:10:58.121464166 +0000 UTC m=+649.016314806"
Jul 14 01:10:58.133004 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:10:58.132970    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-s92qp" podStartSLOduration=1.444106604 podStartE2EDuration="5.132958205s" totalImagesPullingTime="3.688851601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:10:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:10:58.132031351 +0000 UTC m=+649.026881999" watchObservedRunningTime="2026-07-14 01:10:58.132958205 +0000 UTC m=+649.027808844"
Jul 14 01:11:09.122906 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:09.122877    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:11:09.734787 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:09.734755    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"]
Jul 14 01:11:09.734999 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:09.734952    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b" podUID="f5ed26e5-d3b1-4b52-b87c-fa59b0287959" containerName="limitador" containerID="cri-o://1c6f916257f7e5d9d57df85899937444eb851bf211705f4d956a7feea751df2a" gracePeriod=30
Jul 14 01:11:10.173213 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.173171    2580 generic.go:356] "Generic (PLEG): container finished" podID="f5ed26e5-d3b1-4b52-b87c-fa59b0287959" containerID="1c6f916257f7e5d9d57df85899937444eb851bf211705f4d956a7feea751df2a" exitCode=0
Jul 14 01:11:10.173575 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.173241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b" event={"ID":"f5ed26e5-d3b1-4b52-b87c-fa59b0287959","Type":"ContainerDied","Data":"1c6f916257f7e5d9d57df85899937444eb851bf211705f4d956a7feea751df2a"}
Jul 14 01:11:10.283285 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.283265    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:11:10.351081 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.351004    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-config-file\") pod \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\" (UID: \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\") "
Jul 14 01:11:10.351081 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.351074    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tmlb\" (UniqueName: \"kubernetes.io/projected/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-kube-api-access-5tmlb\") pod \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\" (UID: \"f5ed26e5-d3b1-4b52-b87c-fa59b0287959\") "
Jul 14 01:11:10.351345 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.351324    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-config-file" (OuterVolumeSpecName: "config-file") pod "f5ed26e5-d3b1-4b52-b87c-fa59b0287959" (UID: "f5ed26e5-d3b1-4b52-b87c-fa59b0287959"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:11:10.353312 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.353283    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-kube-api-access-5tmlb" (OuterVolumeSpecName: "kube-api-access-5tmlb") pod "f5ed26e5-d3b1-4b52-b87c-fa59b0287959" (UID: "f5ed26e5-d3b1-4b52-b87c-fa59b0287959"). InnerVolumeSpecName "kube-api-access-5tmlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:11:10.452210 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.452169    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tmlb\" (UniqueName: \"kubernetes.io/projected/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-kube-api-access-5tmlb\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:11:10.452210 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:10.452208    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f5ed26e5-d3b1-4b52-b87c-fa59b0287959-config-file\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:11:11.179369 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:11.179340    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"
Jul 14 01:11:11.179369 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:11.179357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8xj2b" event={"ID":"f5ed26e5-d3b1-4b52-b87c-fa59b0287959","Type":"ContainerDied","Data":"ab398bd4600ccb659543d9b10fbb3bbca013c33bd2ccca849c2a100bba5b04ee"}
Jul 14 01:11:11.179923 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:11.179423    2580 scope.go:117] "RemoveContainer" containerID="1c6f916257f7e5d9d57df85899937444eb851bf211705f4d956a7feea751df2a"
Jul 14 01:11:11.199122 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:11.199098    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"]
Jul 14 01:11:11.202362 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:11.202343    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8xj2b"]
Jul 14 01:11:11.642790 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:11.642760    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5ed26e5-d3b1-4b52-b87c-fa59b0287959" path="/var/lib/kubelet/pods/f5ed26e5-d3b1-4b52-b87c-fa59b0287959/volumes"
Jul 14 01:11:14.453570 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.453513    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"]
Jul 14 01:11:14.454067 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.454049    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5ed26e5-d3b1-4b52-b87c-fa59b0287959" containerName="limitador"
Jul 14 01:11:14.454140 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.454071    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5ed26e5-d3b1-4b52-b87c-fa59b0287959" containerName="limitador"
Jul 14 01:11:14.454194 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.454159    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5ed26e5-d3b1-4b52-b87c-fa59b0287959" containerName="limitador"
Jul 14 01:11:14.458547 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.458510    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:14.461051 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.461027    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-gkh6s\""
Jul 14 01:11:14.466672 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.466652    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 01:11:14.466779 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.466744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 14 01:11:14.466838 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.466782    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 01:11:14.473561 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.473541    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"]
Jul 14 01:11:14.589071 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.589042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/cc82c55a-c368-49a3-96a6-a08d379e2a98-data\") pod \"postgres-69fb4594cf-7kmrm\" (UID: \"cc82c55a-c368-49a3-96a6-a08d379e2a98\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:14.589249 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.589159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v6gb\" (UniqueName: \"kubernetes.io/projected/cc82c55a-c368-49a3-96a6-a08d379e2a98-kube-api-access-9v6gb\") pod \"postgres-69fb4594cf-7kmrm\" (UID: \"cc82c55a-c368-49a3-96a6-a08d379e2a98\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:14.690125 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.690092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/cc82c55a-c368-49a3-96a6-a08d379e2a98-data\") pod \"postgres-69fb4594cf-7kmrm\" (UID: \"cc82c55a-c368-49a3-96a6-a08d379e2a98\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:14.690284 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.690180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v6gb\" (UniqueName: \"kubernetes.io/projected/cc82c55a-c368-49a3-96a6-a08d379e2a98-kube-api-access-9v6gb\") pod \"postgres-69fb4594cf-7kmrm\" (UID: \"cc82c55a-c368-49a3-96a6-a08d379e2a98\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:14.690460 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.690441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/cc82c55a-c368-49a3-96a6-a08d379e2a98-data\") pod \"postgres-69fb4594cf-7kmrm\" (UID: \"cc82c55a-c368-49a3-96a6-a08d379e2a98\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:14.700319 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.700297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v6gb\" (UniqueName: \"kubernetes.io/projected/cc82c55a-c368-49a3-96a6-a08d379e2a98-kube-api-access-9v6gb\") pod \"postgres-69fb4594cf-7kmrm\" (UID: \"cc82c55a-c368-49a3-96a6-a08d379e2a98\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:14.769330 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.769304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:14.901869 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:14.901836    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"]
Jul 14 01:11:14.905184 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:11:14.905156    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc82c55a_c368_49a3_96a6_a08d379e2a98.slice/crio-1bb7244205cab94a6621530f8b93c63e7e80eae37f22fbb6c84fc0b52b506cdd WatchSource:0}: Error finding container 1bb7244205cab94a6621530f8b93c63e7e80eae37f22fbb6c84fc0b52b506cdd: Status 404 returned error can't find the container with id 1bb7244205cab94a6621530f8b93c63e7e80eae37f22fbb6c84fc0b52b506cdd
Jul 14 01:11:15.202581 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:15.202465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm" event={"ID":"cc82c55a-c368-49a3-96a6-a08d379e2a98","Type":"ContainerStarted","Data":"1bb7244205cab94a6621530f8b93c63e7e80eae37f22fbb6c84fc0b52b506cdd"}
Jul 14 01:11:21.236975 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:21.236926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm" event={"ID":"cc82c55a-c368-49a3-96a6-a08d379e2a98","Type":"ContainerStarted","Data":"53ffade155956e71a4aac2b08731267210657e7f0cad7e6cff9047140d900c5e"}
Jul 14 01:11:21.237335 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:21.237100    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:21.252849 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:21.252808    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm" podStartSLOduration=1.650369098 podStartE2EDuration="7.25279431s" totalImagesPullingTime="5.602425212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:11:21.250486808 +0000 UTC m=+672.145337446" watchObservedRunningTime="2026-07-14 01:11:21.25279431 +0000 UTC m=+672.147644949"
Jul 14 01:11:27.272178 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:27.272148    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7kmrm"
Jul 14 01:11:30.879510 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:30.879472    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s92qp"]
Jul 14 01:11:30.880076 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:30.879749    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-s92qp" podUID="424265bf-371a-4549-a93f-769b82de00b4" containerName="authorino" containerID="cri-o://36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30" gracePeriod=30
Jul 14 01:11:31.166692 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.166566    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-s92qp"
Jul 14 01:11:31.242948 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.242921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwbpl\" (UniqueName: \"kubernetes.io/projected/424265bf-371a-4549-a93f-769b82de00b4-kube-api-access-pwbpl\") pod \"424265bf-371a-4549-a93f-769b82de00b4\" (UID: \"424265bf-371a-4549-a93f-769b82de00b4\") "
Jul 14 01:11:31.245102 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.245075    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/424265bf-371a-4549-a93f-769b82de00b4-kube-api-access-pwbpl" (OuterVolumeSpecName: "kube-api-access-pwbpl") pod "424265bf-371a-4549-a93f-769b82de00b4" (UID: "424265bf-371a-4549-a93f-769b82de00b4"). InnerVolumeSpecName "kube-api-access-pwbpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:11:31.285449 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.285419    2580 generic.go:356] "Generic (PLEG): container finished" podID="424265bf-371a-4549-a93f-769b82de00b4" containerID="36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30" exitCode=0
Jul 14 01:11:31.285586 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.285458    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-s92qp"
Jul 14 01:11:31.285586 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.285488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-s92qp" event={"ID":"424265bf-371a-4549-a93f-769b82de00b4","Type":"ContainerDied","Data":"36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30"}
Jul 14 01:11:31.285586 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.285536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-s92qp" event={"ID":"424265bf-371a-4549-a93f-769b82de00b4","Type":"ContainerDied","Data":"cdd194857b2964d1c95b7b3424a9851098361816300403628155768ed1e97014"}
Jul 14 01:11:31.285586 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.285553    2580 scope.go:117] "RemoveContainer" containerID="36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30"
Jul 14 01:11:31.294396 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.294375    2580 scope.go:117] "RemoveContainer" containerID="36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30"
Jul 14 01:11:31.294673 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:11:31.294655    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30\": container with ID starting with 36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30 not found: ID does not exist" containerID="36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30"
Jul 14 01:11:31.294712 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.294682    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30"} err="failed to get container status \"36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30\": rpc error: code = NotFound desc = could not find container \"36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30\": container with ID starting with 36656772be7d4aa4ba544d9b83b392ea4ac2f21958daea632a5500bbb2ad9e30 not found: ID does not exist"
Jul 14 01:11:31.303869 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.303840    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s92qp"]
Jul 14 01:11:31.307132 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.307111    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s92qp"]
Jul 14 01:11:31.343601 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.343576    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwbpl\" (UniqueName: \"kubernetes.io/projected/424265bf-371a-4549-a93f-769b82de00b4-kube-api-access-pwbpl\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:11:31.642510 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:31.642477    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="424265bf-371a-4549-a93f-769b82de00b4" path="/var/lib/kubelet/pods/424265bf-371a-4549-a93f-769b82de00b4/volumes"
Jul 14 01:11:35.882976 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.882945    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75868574bf-9b48f"]
Jul 14 01:11:35.883575 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.883292    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424265bf-371a-4549-a93f-769b82de00b4" containerName="authorino"
Jul 14 01:11:35.883575 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.883302    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="424265bf-371a-4549-a93f-769b82de00b4" containerName="authorino"
Jul 14 01:11:35.883575 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.883361    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="424265bf-371a-4549-a93f-769b82de00b4" containerName="authorino"
Jul 14 01:11:35.888016 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.887997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:35.889917 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.889899    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 14 01:11:35.889917 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.889913    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 14 01:11:35.890064 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.889961    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zrmp8\""
Jul 14 01:11:35.894467 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.894446    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75868574bf-9b48f"]
Jul 14 01:11:35.987557 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.987504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4c705ab-a70c-4080-9757-d2542a9b6c76-webhook-cert\") pod \"maas-controller-75868574bf-9b48f\" (UID: \"d4c705ab-a70c-4080-9757-d2542a9b6c76\") " pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:35.987557 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:35.987561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jfks\" (UniqueName: \"kubernetes.io/projected/d4c705ab-a70c-4080-9757-d2542a9b6c76-kube-api-access-8jfks\") pod \"maas-controller-75868574bf-9b48f\" (UID: \"d4c705ab-a70c-4080-9757-d2542a9b6c76\") " pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:36.088748 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:36.088712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4c705ab-a70c-4080-9757-d2542a9b6c76-webhook-cert\") pod \"maas-controller-75868574bf-9b48f\" (UID: \"d4c705ab-a70c-4080-9757-d2542a9b6c76\") " pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:36.088748 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:36.088753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jfks\" (UniqueName: \"kubernetes.io/projected/d4c705ab-a70c-4080-9757-d2542a9b6c76-kube-api-access-8jfks\") pod \"maas-controller-75868574bf-9b48f\" (UID: \"d4c705ab-a70c-4080-9757-d2542a9b6c76\") " pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:36.091417 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:36.091397    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4c705ab-a70c-4080-9757-d2542a9b6c76-webhook-cert\") pod \"maas-controller-75868574bf-9b48f\" (UID: \"d4c705ab-a70c-4080-9757-d2542a9b6c76\") " pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:36.095995 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:36.095977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jfks\" (UniqueName: \"kubernetes.io/projected/d4c705ab-a70c-4080-9757-d2542a9b6c76-kube-api-access-8jfks\") pod \"maas-controller-75868574bf-9b48f\" (UID: \"d4c705ab-a70c-4080-9757-d2542a9b6c76\") " pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:36.200845 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:36.200756    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:36.537389 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:36.537361    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75868574bf-9b48f"]
Jul 14 01:11:36.539912 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:11:36.539882    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c705ab_a70c_4080_9757_d2542a9b6c76.slice/crio-a27257121a3102b0cf20e4879b06ea2760ea7b9bda2c1f169e288e3e96157710 WatchSource:0}: Error finding container a27257121a3102b0cf20e4879b06ea2760ea7b9bda2c1f169e288e3e96157710: Status 404 returned error can't find the container with id a27257121a3102b0cf20e4879b06ea2760ea7b9bda2c1f169e288e3e96157710
Jul 14 01:11:37.319053 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:37.319015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75868574bf-9b48f" event={"ID":"d4c705ab-a70c-4080-9757-d2542a9b6c76","Type":"ContainerStarted","Data":"a27257121a3102b0cf20e4879b06ea2760ea7b9bda2c1f169e288e3e96157710"}
Jul 14 01:11:39.330287 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:39.330192    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75868574bf-9b48f" event={"ID":"d4c705ab-a70c-4080-9757-d2542a9b6c76","Type":"ContainerStarted","Data":"1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489"}
Jul 14 01:11:39.330775 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:39.330308    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:11:39.347065 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:39.347017    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75868574bf-9b48f" podStartSLOduration=1.867453027 podStartE2EDuration="4.347000241s" totalImagesPullingTime="2.479547214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:11:39.346213168 +0000 UTC m=+690.241063809" watchObservedRunningTime="2026-07-14 01:11:39.347000241 +0000 UTC m=+690.241850880"
Jul 14 01:11:50.336388 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:11:50.336354    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:12:14.134742 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.134714    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"]
Jul 14 01:12:14.139432 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.139413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.141668 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.141646    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 14 01:12:14.141766 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.141673    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8hlqx\""
Jul 14 01:12:14.147712 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.147691    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"]
Jul 14 01:12:14.237284 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.237245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e427c6-0e76-41c2-8eed-c5a98f65d954-openshift-service-ca\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.237284 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.237279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84e427c6-0e76-41c2-8eed-c5a98f65d954-maas-api-tls\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.237485 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.237416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f2m8\" (UniqueName: \"kubernetes.io/projected/84e427c6-0e76-41c2-8eed-c5a98f65d954-kube-api-access-2f2m8\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.338755 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.338720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f2m8\" (UniqueName: \"kubernetes.io/projected/84e427c6-0e76-41c2-8eed-c5a98f65d954-kube-api-access-2f2m8\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.338926 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.338768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e427c6-0e76-41c2-8eed-c5a98f65d954-openshift-service-ca\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.338926 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.338790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84e427c6-0e76-41c2-8eed-c5a98f65d954-maas-api-tls\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.339414 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.339387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e427c6-0e76-41c2-8eed-c5a98f65d954-openshift-service-ca\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.341404 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.341383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84e427c6-0e76-41c2-8eed-c5a98f65d954-maas-api-tls\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.346357 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.346336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f2m8\" (UniqueName: \"kubernetes.io/projected/84e427c6-0e76-41c2-8eed-c5a98f65d954-kube-api-access-2f2m8\") pod \"maas-api-74f8fc7886-92hx5\" (UID: \"84e427c6-0e76-41c2-8eed-c5a98f65d954\") " pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.451245 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.451168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:14.637736 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.637709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"]
Jul 14 01:12:14.640478 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:12:14.640454    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e427c6_0e76_41c2_8eed_c5a98f65d954.slice/crio-856fd6a17c7a3e6945ce7466b94692c7560fa5422dd1a0508e92f2199927846f WatchSource:0}: Error finding container 856fd6a17c7a3e6945ce7466b94692c7560fa5422dd1a0508e92f2199927846f: Status 404 returned error can't find the container with id 856fd6a17c7a3e6945ce7466b94692c7560fa5422dd1a0508e92f2199927846f
Jul 14 01:12:14.830968 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.830937    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"]
Jul 14 01:12:14.833939 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.833921    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:14.835916 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.835890    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 14 01:12:14.836024 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.835890    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wvnc9\""
Jul 14 01:12:14.841729 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.841707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"]
Jul 14 01:12:14.842903 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.842874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4583f204-ce27-431d-8e2a-2fc74b67fd08-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ggjh2\" (UID: \"4583f204-ce27-431d-8e2a-2fc74b67fd08\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:14.843027 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.842918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2c8l\" (UniqueName: \"kubernetes.io/projected/4583f204-ce27-431d-8e2a-2fc74b67fd08-kube-api-access-v2c8l\") pod \"payload-processing-f9b8d9b9b-ggjh2\" (UID: \"4583f204-ce27-431d-8e2a-2fc74b67fd08\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:14.944395 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.944363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4583f204-ce27-431d-8e2a-2fc74b67fd08-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ggjh2\" (UID: \"4583f204-ce27-431d-8e2a-2fc74b67fd08\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:14.944395 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.944399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2c8l\" (UniqueName: \"kubernetes.io/projected/4583f204-ce27-431d-8e2a-2fc74b67fd08-kube-api-access-v2c8l\") pod \"payload-processing-f9b8d9b9b-ggjh2\" (UID: \"4583f204-ce27-431d-8e2a-2fc74b67fd08\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:14.945069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.945050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4583f204-ce27-431d-8e2a-2fc74b67fd08-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ggjh2\" (UID: \"4583f204-ce27-431d-8e2a-2fc74b67fd08\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:14.951717 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:14.951694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2c8l\" (UniqueName: \"kubernetes.io/projected/4583f204-ce27-431d-8e2a-2fc74b67fd08-kube-api-access-v2c8l\") pod \"payload-processing-f9b8d9b9b-ggjh2\" (UID: \"4583f204-ce27-431d-8e2a-2fc74b67fd08\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:15.146319 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:15.146234    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:15.278989 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:15.278876    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"]
Jul 14 01:12:15.281585 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:12:15.281552    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4583f204_ce27_431d_8e2a_2fc74b67fd08.slice/crio-8c94e932878dad1be586e8a46878dbed8c80df199b309cd5dd29cf016f3412ee WatchSource:0}: Error finding container 8c94e932878dad1be586e8a46878dbed8c80df199b309cd5dd29cf016f3412ee: Status 404 returned error can't find the container with id 8c94e932878dad1be586e8a46878dbed8c80df199b309cd5dd29cf016f3412ee
Jul 14 01:12:15.508470 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:15.508434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5" event={"ID":"84e427c6-0e76-41c2-8eed-c5a98f65d954","Type":"ContainerStarted","Data":"856fd6a17c7a3e6945ce7466b94692c7560fa5422dd1a0508e92f2199927846f"}
Jul 14 01:12:15.509664 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:15.509642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2" event={"ID":"4583f204-ce27-431d-8e2a-2fc74b67fd08","Type":"ContainerStarted","Data":"8c94e932878dad1be586e8a46878dbed8c80df199b309cd5dd29cf016f3412ee"}
Jul 14 01:12:17.526355 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:17.526311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2" event={"ID":"4583f204-ce27-431d-8e2a-2fc74b67fd08","Type":"ContainerStarted","Data":"471b6088edf153f14cff53803cb11600148732434f7d62ee03f908eb2d72cf3b"}
Jul 14 01:12:17.526834 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:17.526428    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:12:17.527832 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:17.527809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5" event={"ID":"84e427c6-0e76-41c2-8eed-c5a98f65d954","Type":"ContainerStarted","Data":"3e56136df361e824fc300eaf0061c9c9db8f893f3db9bcafa3d83c9996c102b7"}
Jul 14 01:12:17.527967 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:17.527946    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:17.541962 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:17.541924    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2" podStartSLOduration=1.939332281 podStartE2EDuration="3.541910442s" totalImagesPullingTime="1.602578161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:12:17.540150592 +0000 UTC m=+728.435001237" watchObservedRunningTime="2026-07-14 01:12:17.541910442 +0000 UTC m=+728.436761081"
Jul 14 01:12:17.554726 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:17.554679    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5" podStartSLOduration=1.308211772 podStartE2EDuration="3.554664658s" totalImagesPullingTime="2.246452886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:12:17.552981211 +0000 UTC m=+728.447831875" watchObservedRunningTime="2026-07-14 01:12:17.554664658 +0000 UTC m=+728.449515296"
Jul 14 01:12:23.540725 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:23.540697    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-74f8fc7886-92hx5"
Jul 14 01:12:28.537062 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:12:28.537030    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-ggjh2"
Jul 14 01:13:15.858170 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.858136    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"]
Jul 14 01:13:15.861618 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.861594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:15.864007 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.863966    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 01:13:15.864140 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.864021    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 14 01:13:15.864140 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.864085    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vmjgp\""
Jul 14 01:13:15.864140 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.863966    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 01:13:15.869902 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.869883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"]
Jul 14 01:13:15.990156 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.990123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:15.990156 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.990158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:15.990355 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.990203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:15.990355 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.990262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a185c7-ef98-44ee-8fba-17a3aa80a430-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:15.990355 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.990284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brgvf\" (UniqueName: \"kubernetes.io/projected/d0a185c7-ef98-44ee-8fba-17a3aa80a430-kube-api-access-brgvf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:15.990355 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:15.990304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.091502 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.091471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.091691 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.091544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a185c7-ef98-44ee-8fba-17a3aa80a430-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.091691 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.091566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brgvf\" (UniqueName: \"kubernetes.io/projected/d0a185c7-ef98-44ee-8fba-17a3aa80a430-kube-api-access-brgvf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.091691 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.091588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.091846 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.091744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.091846 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.091782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.092009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.091988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.092065 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.092011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.092099 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.092076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.094232 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.094205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a185c7-ef98-44ee-8fba-17a3aa80a430-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.094232 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.094216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0a185c7-ef98-44ee-8fba-17a3aa80a430-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.098811 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.098791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brgvf\" (UniqueName: \"kubernetes.io/projected/d0a185c7-ef98-44ee-8fba-17a3aa80a430-kube-api-access-brgvf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-88sfn\" (UID: \"d0a185c7-ef98-44ee-8fba-17a3aa80a430\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.173638 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.173562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:16.309482 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.309463    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"]
Jul 14 01:13:16.312135 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:13:16.312110    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a185c7_ef98_44ee_8fba_17a3aa80a430.slice/crio-3afc558271cfe34224d0be809a33bf7ea1d0c6760cd81a8bb0651aa228ad059e WatchSource:0}: Error finding container 3afc558271cfe34224d0be809a33bf7ea1d0c6760cd81a8bb0651aa228ad059e: Status 404 returned error can't find the container with id 3afc558271cfe34224d0be809a33bf7ea1d0c6760cd81a8bb0651aa228ad059e
Jul 14 01:13:16.807465 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:16.807381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn" event={"ID":"d0a185c7-ef98-44ee-8fba-17a3aa80a430","Type":"ContainerStarted","Data":"3afc558271cfe34224d0be809a33bf7ea1d0c6760cd81a8bb0651aa228ad059e"}
Jul 14 01:13:17.813221 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:17.813185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn" event={"ID":"d0a185c7-ef98-44ee-8fba-17a3aa80a430","Type":"ContainerStarted","Data":"64344f03202530c94fba4257ba902c642a9565ff76fe15a9da5344fb881678f6"}
Jul 14 01:13:17.828791 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:17.828736    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn" podStartSLOduration=1.800866597 podStartE2EDuration="2.828719917s" totalImagesPullingTime="1.02785332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:13:17.827352426 +0000 UTC m=+788.722203069" watchObservedRunningTime="2026-07-14 01:13:17.828719917 +0000 UTC m=+788.723570555"
Jul 14 01:13:23.847832 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.847797    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"]
Jul 14 01:13:23.853452 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.853429    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:23.855531 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.855489    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"]
Jul 14 01:13:23.855663 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.855637    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 14 01:13:23.961157 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.961127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:23.961324 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.961182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbj2r\" (UniqueName: \"kubernetes.io/projected/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-kube-api-access-pbj2r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:23.961324 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.961213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:23.961324 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.961233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:23.961324 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.961273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:23.961478 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:23.961329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.061989 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.061955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.062139 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.062023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbj2r\" (UniqueName: \"kubernetes.io/projected/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-kube-api-access-pbj2r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.062139 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.062056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.062139 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.062082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.062139 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.062129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.062313 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.062167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.062492 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.062471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.062601 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.062491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.062601 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.062592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.064408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.064388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.064561 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.064543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.069966 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.069943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbj2r\" (UniqueName: \"kubernetes.io/projected/7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1-kube-api-access-pbj2r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx\" (UID: \"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.168856 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.168777    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:24.300499 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.300474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"]
Jul 14 01:13:24.303164 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:13:24.303126    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa8ce0d_68d0_474a_85fb_2a8d46ff70f1.slice/crio-5bce2c7e8d3a708c8ad12d180a68cc2f29184d759862e6b4dbe277d2c77d912b WatchSource:0}: Error finding container 5bce2c7e8d3a708c8ad12d180a68cc2f29184d759862e6b4dbe277d2c77d912b: Status 404 returned error can't find the container with id 5bce2c7e8d3a708c8ad12d180a68cc2f29184d759862e6b4dbe277d2c77d912b
Jul 14 01:13:24.859948 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.859912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx" event={"ID":"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1","Type":"ContainerStarted","Data":"52abdfce7f71c8330d99c3b558e196146348ec8953e0ff99bd4cd79f0ee9331f"}
Jul 14 01:13:24.860347 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.859960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx" event={"ID":"7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1","Type":"ContainerStarted","Data":"5bce2c7e8d3a708c8ad12d180a68cc2f29184d759862e6b4dbe277d2c77d912b"}
Jul 14 01:13:24.875538 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:24.875472    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx" podStartSLOduration=1.704852674 podStartE2EDuration="1.875454642s" totalImagesPullingTime="170.601968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:13:24.873374029 +0000 UTC m=+795.768224674" watchObservedRunningTime="2026-07-14 01:13:24.875454642 +0000 UTC m=+795.770305282"
Jul 14 01:13:26.173991 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:26.173952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:26.174370 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:26.174021    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:26.186729 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:26.186708    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:26.881382 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:26.881351    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-88sfn"
Jul 14 01:13:34.169934 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:34.169853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:34.169934 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:34.169922    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:34.182472 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:34.182446    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:13:34.922086 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:13:34.922054    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx"
Jul 14 01:16:02.288226 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.288192    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75868574bf-9b48f"]
Jul 14 01:16:02.288739 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.288409    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75868574bf-9b48f" podUID="d4c705ab-a70c-4080-9757-d2542a9b6c76" containerName="manager" containerID="cri-o://1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489" gracePeriod=30
Jul 14 01:16:02.538144 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.538120    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:16:02.612709 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.612632    2580 generic.go:356] "Generic (PLEG): container finished" podID="d4c705ab-a70c-4080-9757-d2542a9b6c76" containerID="1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489" exitCode=0
Jul 14 01:16:02.612709 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.612679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75868574bf-9b48f" event={"ID":"d4c705ab-a70c-4080-9757-d2542a9b6c76","Type":"ContainerDied","Data":"1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489"}
Jul 14 01:16:02.612884 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.612731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75868574bf-9b48f"
Jul 14 01:16:02.612884 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.612745    2580 scope.go:117] "RemoveContainer" containerID="1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489"
Jul 14 01:16:02.612884 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.612735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75868574bf-9b48f" event={"ID":"d4c705ab-a70c-4080-9757-d2542a9b6c76","Type":"ContainerDied","Data":"a27257121a3102b0cf20e4879b06ea2760ea7b9bda2c1f169e288e3e96157710"}
Jul 14 01:16:02.621121 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.621106    2580 scope.go:117] "RemoveContainer" containerID="1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489"
Jul 14 01:16:02.621381 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:16:02.621359    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489\": container with ID starting with 1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489 not found: ID does not exist" containerID="1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489"
Jul 14 01:16:02.621430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.621386    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489"} err="failed to get container status \"1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489\": rpc error: code = NotFound desc = could not find container \"1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489\": container with ID starting with 1b496db906cb8d23359e1ea7c532ebd49f8611627edbc8b0b6af6fb4a2982489 not found: ID does not exist"
Jul 14 01:16:02.627669 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.627651    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4c705ab-a70c-4080-9757-d2542a9b6c76-webhook-cert\") pod \"d4c705ab-a70c-4080-9757-d2542a9b6c76\" (UID: \"d4c705ab-a70c-4080-9757-d2542a9b6c76\") "
Jul 14 01:16:02.627795 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.627784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jfks\" (UniqueName: \"kubernetes.io/projected/d4c705ab-a70c-4080-9757-d2542a9b6c76-kube-api-access-8jfks\") pod \"d4c705ab-a70c-4080-9757-d2542a9b6c76\" (UID: \"d4c705ab-a70c-4080-9757-d2542a9b6c76\") "
Jul 14 01:16:02.629838 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.629810    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c705ab-a70c-4080-9757-d2542a9b6c76-kube-api-access-8jfks" (OuterVolumeSpecName: "kube-api-access-8jfks") pod "d4c705ab-a70c-4080-9757-d2542a9b6c76" (UID: "d4c705ab-a70c-4080-9757-d2542a9b6c76"). InnerVolumeSpecName "kube-api-access-8jfks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:16:02.629919 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.629859    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4c705ab-a70c-4080-9757-d2542a9b6c76-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d4c705ab-a70c-4080-9757-d2542a9b6c76" (UID: "d4c705ab-a70c-4080-9757-d2542a9b6c76"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:16:02.728923 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.728897    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jfks\" (UniqueName: \"kubernetes.io/projected/d4c705ab-a70c-4080-9757-d2542a9b6c76-kube-api-access-8jfks\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:16:02.728923 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.728924    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4c705ab-a70c-4080-9757-d2542a9b6c76-webhook-cert\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:16:02.932755 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.932724    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75868574bf-9b48f"]
Jul 14 01:16:02.935929 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:02.935908    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75868574bf-9b48f"]
Jul 14 01:16:03.642446 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.642414    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c705ab-a70c-4080-9757-d2542a9b6c76" path="/var/lib/kubelet/pods/d4c705ab-a70c-4080-9757-d2542a9b6c76/volumes"
Jul 14 01:16:03.726365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.726332    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75868574bf-zknhx"]
Jul 14 01:16:03.726723 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.726711    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c705ab-a70c-4080-9757-d2542a9b6c76" containerName="manager"
Jul 14 01:16:03.726767 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.726725    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c705ab-a70c-4080-9757-d2542a9b6c76" containerName="manager"
Jul 14 01:16:03.726808 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.726798    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c705ab-a70c-4080-9757-d2542a9b6c76" containerName="manager"
Jul 14 01:16:03.731341 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.731322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:03.733785 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.733760    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 14 01:16:03.733785 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.733777    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 14 01:16:03.733937 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.733787    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zrmp8\""
Jul 14 01:16:03.737354 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.737328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75868574bf-zknhx"]
Jul 14 01:16:03.839054 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.839017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dd08212-b3b0-4e6e-b575-b745166eb21e-webhook-cert\") pod \"maas-controller-75868574bf-zknhx\" (UID: \"6dd08212-b3b0-4e6e-b575-b745166eb21e\") " pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:03.839216 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.839080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk6lx\" (UniqueName: \"kubernetes.io/projected/6dd08212-b3b0-4e6e-b575-b745166eb21e-kube-api-access-mk6lx\") pod \"maas-controller-75868574bf-zknhx\" (UID: \"6dd08212-b3b0-4e6e-b575-b745166eb21e\") " pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:03.940485 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.940399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk6lx\" (UniqueName: \"kubernetes.io/projected/6dd08212-b3b0-4e6e-b575-b745166eb21e-kube-api-access-mk6lx\") pod \"maas-controller-75868574bf-zknhx\" (UID: \"6dd08212-b3b0-4e6e-b575-b745166eb21e\") " pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:03.940657 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.940495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dd08212-b3b0-4e6e-b575-b745166eb21e-webhook-cert\") pod \"maas-controller-75868574bf-zknhx\" (UID: \"6dd08212-b3b0-4e6e-b575-b745166eb21e\") " pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:03.942987 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.942961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dd08212-b3b0-4e6e-b575-b745166eb21e-webhook-cert\") pod \"maas-controller-75868574bf-zknhx\" (UID: \"6dd08212-b3b0-4e6e-b575-b745166eb21e\") " pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:03.947343 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:03.947319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk6lx\" (UniqueName: \"kubernetes.io/projected/6dd08212-b3b0-4e6e-b575-b745166eb21e-kube-api-access-mk6lx\") pod \"maas-controller-75868574bf-zknhx\" (UID: \"6dd08212-b3b0-4e6e-b575-b745166eb21e\") " pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:04.043016 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:04.042989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:04.172821 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:04.172795    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75868574bf-zknhx"]
Jul 14 01:16:04.175578 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:16:04.175548    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd08212_b3b0_4e6e_b575_b745166eb21e.slice/crio-330ec99eed9c3fddfb5927320233e541e67e0e1ab591621ca191dfd484c8a1ff WatchSource:0}: Error finding container 330ec99eed9c3fddfb5927320233e541e67e0e1ab591621ca191dfd484c8a1ff: Status 404 returned error can't find the container with id 330ec99eed9c3fddfb5927320233e541e67e0e1ab591621ca191dfd484c8a1ff
Jul 14 01:16:04.177168 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:04.177149    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 01:16:04.625302 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:04.625269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75868574bf-zknhx" event={"ID":"6dd08212-b3b0-4e6e-b575-b745166eb21e","Type":"ContainerStarted","Data":"330ec99eed9c3fddfb5927320233e541e67e0e1ab591621ca191dfd484c8a1ff"}
Jul 14 01:16:05.631626 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:05.631591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75868574bf-zknhx" event={"ID":"6dd08212-b3b0-4e6e-b575-b745166eb21e","Type":"ContainerStarted","Data":"62807eac1002151ce5f863ab7bf675bfc049e0c57d01769e3ef6db5a3abf4403"}
Jul 14 01:16:05.632055 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:05.631689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:16:05.646639 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:05.646588    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75868574bf-zknhx" podStartSLOduration=2.12825967 podStartE2EDuration="2.646571933s" totalImagesPullingTime="518.312263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:16:05.644813488 +0000 UTC m=+956.539664127" watchObservedRunningTime="2026-07-14 01:16:05.646571933 +0000 UTC m=+956.541422571"
Jul 14 01:16:16.638216 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:16:16.638184    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75868574bf-zknhx"
Jul 14 01:28:29.169736 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.169705    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"]
Jul 14 01:28:29.173413 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.173394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.175491 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.175471    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-62a42e82-serving-cert\""
Jul 14 01:28:29.182859 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.182839    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"]
Jul 14 01:28:29.253953 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.253927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a490847-36a0-49ca-8e72-e5d23af3f296-maas-api-tls\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.254099 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.253987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a490847-36a0-49ca-8e72-e5d23af3f296-openshift-service-ca\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.254099 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.254007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgzc8\" (UniqueName: \"kubernetes.io/projected/2a490847-36a0-49ca-8e72-e5d23af3f296-kube-api-access-jgzc8\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.355243 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.355214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a490847-36a0-49ca-8e72-e5d23af3f296-maas-api-tls\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.355390 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.355307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a490847-36a0-49ca-8e72-e5d23af3f296-openshift-service-ca\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.355390 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.355339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgzc8\" (UniqueName: \"kubernetes.io/projected/2a490847-36a0-49ca-8e72-e5d23af3f296-kube-api-access-jgzc8\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.355976 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.355948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a490847-36a0-49ca-8e72-e5d23af3f296-openshift-service-ca\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.357905 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.357885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a490847-36a0-49ca-8e72-e5d23af3f296-maas-api-tls\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.368320 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.368303    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgzc8\" (UniqueName: \"kubernetes.io/projected/2a490847-36a0-49ca-8e72-e5d23af3f296-kube-api-access-jgzc8\") pod \"maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.486574 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.486485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:29.619620 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.619592    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"]
Jul 14 01:28:29.621848 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:28:29.621820    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a490847_36a0_49ca_8e72_e5d23af3f296.slice/crio-4ef8d93ff875277dc4818106fd95e517a4adf65eb42bef9af58e6b4e72692e1b WatchSource:0}: Error finding container 4ef8d93ff875277dc4818106fd95e517a4adf65eb42bef9af58e6b4e72692e1b: Status 404 returned error can't find the container with id 4ef8d93ff875277dc4818106fd95e517a4adf65eb42bef9af58e6b4e72692e1b
Jul 14 01:28:29.623218 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:29.623201    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 01:28:30.120669 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:30.120640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2" event={"ID":"2a490847-36a0-49ca-8e72-e5d23af3f296","Type":"ContainerStarted","Data":"4ef8d93ff875277dc4818106fd95e517a4adf65eb42bef9af58e6b4e72692e1b"}
Jul 14 01:28:31.127044 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:31.127013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2" event={"ID":"2a490847-36a0-49ca-8e72-e5d23af3f296","Type":"ContainerStarted","Data":"61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a"}
Jul 14 01:28:31.127493 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:31.127070    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:28:31.143476 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:31.143425    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2" podStartSLOduration=1.6914213139999998 podStartE2EDuration="2.143405514s" totalImagesPullingTime="451.9842ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:28:31.142095585 +0000 UTC m=+1702.036946226" watchObservedRunningTime="2026-07-14 01:28:31.143405514 +0000 UTC m=+1702.038256153"
Jul 14 01:28:37.137082 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:28:37.137051    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:29:44.617020 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:44.616985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"]
Jul 14 01:29:44.617488 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:44.617270    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2" podUID="2a490847-36a0-49ca-8e72-e5d23af3f296" containerName="maas-api" containerID="cri-o://61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a" gracePeriod=30
Jul 14 01:29:44.868583 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:44.868509    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:29:45.057814 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.057781    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a490847-36a0-49ca-8e72-e5d23af3f296-openshift-service-ca\") pod \"2a490847-36a0-49ca-8e72-e5d23af3f296\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") "
Jul 14 01:29:45.058017 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.057900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgzc8\" (UniqueName: \"kubernetes.io/projected/2a490847-36a0-49ca-8e72-e5d23af3f296-kube-api-access-jgzc8\") pod \"2a490847-36a0-49ca-8e72-e5d23af3f296\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") "
Jul 14 01:29:45.058017 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.057951    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a490847-36a0-49ca-8e72-e5d23af3f296-maas-api-tls\") pod \"2a490847-36a0-49ca-8e72-e5d23af3f296\" (UID: \"2a490847-36a0-49ca-8e72-e5d23af3f296\") "
Jul 14 01:29:45.058225 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.058196    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a490847-36a0-49ca-8e72-e5d23af3f296-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a490847-36a0-49ca-8e72-e5d23af3f296" (UID: "2a490847-36a0-49ca-8e72-e5d23af3f296"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:29:45.060250 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.060226    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a490847-36a0-49ca-8e72-e5d23af3f296-kube-api-access-jgzc8" (OuterVolumeSpecName: "kube-api-access-jgzc8") pod "2a490847-36a0-49ca-8e72-e5d23af3f296" (UID: "2a490847-36a0-49ca-8e72-e5d23af3f296"). InnerVolumeSpecName "kube-api-access-jgzc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:29:45.060348 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.060264    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a490847-36a0-49ca-8e72-e5d23af3f296-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a490847-36a0-49ca-8e72-e5d23af3f296" (UID: "2a490847-36a0-49ca-8e72-e5d23af3f296"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:29:45.159020 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.158943    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a490847-36a0-49ca-8e72-e5d23af3f296-maas-api-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:29:45.159020 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.158969    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a490847-36a0-49ca-8e72-e5d23af3f296-openshift-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:29:45.159020 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.158978    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgzc8\" (UniqueName: \"kubernetes.io/projected/2a490847-36a0-49ca-8e72-e5d23af3f296-kube-api-access-jgzc8\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:29:45.495613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.495581    2580 generic.go:356] "Generic (PLEG): container finished" podID="2a490847-36a0-49ca-8e72-e5d23af3f296" containerID="61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a" exitCode=0
Jul 14 01:29:45.495773 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.495625    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"
Jul 14 01:29:45.495773 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.495635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2" event={"ID":"2a490847-36a0-49ca-8e72-e5d23af3f296","Type":"ContainerDied","Data":"61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a"}
Jul 14 01:29:45.495773 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.495708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2" event={"ID":"2a490847-36a0-49ca-8e72-e5d23af3f296","Type":"ContainerDied","Data":"4ef8d93ff875277dc4818106fd95e517a4adf65eb42bef9af58e6b4e72692e1b"}
Jul 14 01:29:45.495773 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.495741    2580 scope.go:117] "RemoveContainer" containerID="61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a"
Jul 14 01:29:45.506098 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.506082    2580 scope.go:117] "RemoveContainer" containerID="61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a"
Jul 14 01:29:45.506357 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:29:45.506341    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a\": container with ID starting with 61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a not found: ID does not exist" containerID="61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a"
Jul 14 01:29:45.506402 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.506364    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a"} err="failed to get container status \"61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a\": rpc error: code = NotFound desc = could not find container \"61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a\": container with ID starting with 61ba14f1e50a76181d5d4872711553b68a8ad287f4a48f6ea1f7e800f506011a not found: ID does not exist"
Jul 14 01:29:45.516058 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.516039    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"]
Jul 14 01:29:45.519616 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.519596    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-62a42e82-6f6f666487-wv5r2"]
Jul 14 01:29:45.643107 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:29:45.643077    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a490847-36a0-49ca-8e72-e5d23af3f296" path="/var/lib/kubelet/pods/2a490847-36a0-49ca-8e72-e5d23af3f296/volumes"
Jul 14 01:30:00.137180 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.137146    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm"]
Jul 14 01:30:00.137560 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.137502    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a490847-36a0-49ca-8e72-e5d23af3f296" containerName="maas-api"
Jul 14 01:30:00.137560 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.137513    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a490847-36a0-49ca-8e72-e5d23af3f296" containerName="maas-api"
Jul 14 01:30:00.137642 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.137615    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a490847-36a0-49ca-8e72-e5d23af3f296" containerName="maas-api"
Jul 14 01:30:00.144081 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.144054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm"
Jul 14 01:30:00.147397 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.147369    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:30:00.152704 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.152679    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm"]
Jul 14 01:30:00.158346 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.158325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm"
Jul 14 01:30:00.307015 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.306986    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm"]
Jul 14 01:30:00.309774 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:30:00.309739    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5b0626d_8472_4371_881d_5d04faf65463.slice/crio-49de6103f2d7b7a53bc67f18bddf3bc4d1fb088f84bff4ea083a9c038d295a3e WatchSource:0}: Error finding container 49de6103f2d7b7a53bc67f18bddf3bc4d1fb088f84bff4ea083a9c038d295a3e: Status 404 returned error can't find the container with id 49de6103f2d7b7a53bc67f18bddf3bc4d1fb088f84bff4ea083a9c038d295a3e
Jul 14 01:30:00.580352 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:00.580319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm" event={"ID":"d5b0626d-8472-4371-881d-5d04faf65463","Type":"ContainerStarted","Data":"49de6103f2d7b7a53bc67f18bddf3bc4d1fb088f84bff4ea083a9c038d295a3e"}
Jul 14 01:30:01.538876 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.538713    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"]
Jul 14 01:30:01.542889 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.542861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:01.544785 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.544760    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-7149234b-serving-cert\""
Jul 14 01:30:01.548953 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.548898    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"]
Jul 14 01:30:01.617376 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.617342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a95e21-fd15-4d59-ac88-d01247fa4139-openshift-service-ca\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:01.617661 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.617393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:01.617661 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.617615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbm5q\" (UniqueName: \"kubernetes.io/projected/f3a95e21-fd15-4d59-ac88-d01247fa4139-kube-api-access-dbm5q\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:01.719819 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.718918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbm5q\" (UniqueName: \"kubernetes.io/projected/f3a95e21-fd15-4d59-ac88-d01247fa4139-kube-api-access-dbm5q\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:01.719819 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.719002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a95e21-fd15-4d59-ac88-d01247fa4139-openshift-service-ca\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:01.719819 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.719067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:01.719819 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:30:01.719433    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-serving-cert: secret "maas-api-e2e-migrate-7149234b-serving-cert" not found
Jul 14 01:30:01.719819 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:30:01.719506    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls podName:f3a95e21-fd15-4d59-ac88-d01247fa4139 nodeName:}" failed. No retries permitted until 2026-07-14 01:30:02.21948561 +0000 UTC m=+1793.114336230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls") pod "maas-api-e2e-migrate-7149234b-686c77dc78-msx2b" (UID: "f3a95e21-fd15-4d59-ac88-d01247fa4139") : secret "maas-api-e2e-migrate-7149234b-serving-cert" not found
Jul 14 01:30:01.720247 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.720148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a95e21-fd15-4d59-ac88-d01247fa4139-openshift-service-ca\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:01.728044 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:01.728023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbm5q\" (UniqueName: \"kubernetes.io/projected/f3a95e21-fd15-4d59-ac88-d01247fa4139-kube-api-access-dbm5q\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:02.225980 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:02.225938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:02.228851 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:02.228824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls\") pod \"maas-api-e2e-migrate-7149234b-686c77dc78-msx2b\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:02.458569 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:02.458502    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:02.741750 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:02.741720    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"]
Jul 14 01:30:02.745043 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:30:02.745010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a95e21_fd15_4d59_ac88_d01247fa4139.slice/crio-ee190d92cf4e486ffdc58a773cf58e85426cbe0949db8cce49e5562b704e8a38 WatchSource:0}: Error finding container ee190d92cf4e486ffdc58a773cf58e85426cbe0949db8cce49e5562b704e8a38: Status 404 returned error can't find the container with id ee190d92cf4e486ffdc58a773cf58e85426cbe0949db8cce49e5562b704e8a38
Jul 14 01:30:03.598895 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:03.598857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b" event={"ID":"f3a95e21-fd15-4d59-ac88-d01247fa4139","Type":"ContainerStarted","Data":"ee190d92cf4e486ffdc58a773cf58e85426cbe0949db8cce49e5562b704e8a38"}
Jul 14 01:30:03.600478 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:03.600455    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm" event={"ID":"d5b0626d-8472-4371-881d-5d04faf65463","Type":"ContainerStarted","Data":"73d55e8a94072c54d5f0f480ffc4de5c52553aa7c40036d232d5b4925640fb13"}
Jul 14 01:30:03.616233 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:03.616185    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm" podStartSLOduration=1.261576992 podStartE2EDuration="3.616166141s" totalImagesPullingTime="2.354589149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:30:03.614880592 +0000 UTC m=+1794.509731232" watchObservedRunningTime="2026-07-14 01:30:03.616166141 +0000 UTC m=+1794.511016781"
Jul 14 01:30:04.606773 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:04.606732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b" event={"ID":"f3a95e21-fd15-4d59-ac88-d01247fa4139","Type":"ContainerStarted","Data":"e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682"}
Jul 14 01:30:04.607232 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:04.606803    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:30:04.608200 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:04.608172    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5b0626d-8472-4371-881d-5d04faf65463" containerID="73d55e8a94072c54d5f0f480ffc4de5c52553aa7c40036d232d5b4925640fb13" exitCode=0
Jul 14 01:30:04.608291 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:04.608220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm" event={"ID":"d5b0626d-8472-4371-881d-5d04faf65463","Type":"ContainerDied","Data":"73d55e8a94072c54d5f0f480ffc4de5c52553aa7c40036d232d5b4925640fb13"}
Jul 14 01:30:04.642640 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:04.642590    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b" podStartSLOduration=2.743853449 podStartE2EDuration="3.64256851s" totalImagesPullingTime="898.715061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:30:04.641990438 +0000 UTC m=+1795.536841077" watchObservedRunningTime="2026-07-14 01:30:04.64256851 +0000 UTC m=+1795.537419149"
Jul 14 01:30:05.751901 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:05.751874    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm"
Jul 14 01:30:06.619439 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:06.619410    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm"
Jul 14 01:30:06.619625 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:06.619437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-2zhvm" event={"ID":"d5b0626d-8472-4371-881d-5d04faf65463","Type":"ContainerDied","Data":"49de6103f2d7b7a53bc67f18bddf3bc4d1fb088f84bff4ea083a9c038d295a3e"}
Jul 14 01:30:06.619625 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:06.619487    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49de6103f2d7b7a53bc67f18bddf3bc4d1fb088f84bff4ea083a9c038d295a3e"
Jul 14 01:30:10.618258 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:30:10.618232    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:31:06.385775 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.385737    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"]
Jul 14 01:31:06.386339 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.386000    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b" podUID="f3a95e21-fd15-4d59-ac88-d01247fa4139" containerName="maas-api" containerID="cri-o://e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682" gracePeriod=30
Jul 14 01:31:06.661881 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.661852    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:31:06.813768 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.813736    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbm5q\" (UniqueName: \"kubernetes.io/projected/f3a95e21-fd15-4d59-ac88-d01247fa4139-kube-api-access-dbm5q\") pod \"f3a95e21-fd15-4d59-ac88-d01247fa4139\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") "
Jul 14 01:31:06.813937 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.813802    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a95e21-fd15-4d59-ac88-d01247fa4139-openshift-service-ca\") pod \"f3a95e21-fd15-4d59-ac88-d01247fa4139\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") "
Jul 14 01:31:06.813937 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.813833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls\") pod \"f3a95e21-fd15-4d59-ac88-d01247fa4139\" (UID: \"f3a95e21-fd15-4d59-ac88-d01247fa4139\") "
Jul 14 01:31:06.814203 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.814178    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a95e21-fd15-4d59-ac88-d01247fa4139-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3a95e21-fd15-4d59-ac88-d01247fa4139" (UID: "f3a95e21-fd15-4d59-ac88-d01247fa4139"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:31:06.816037 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.816013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a95e21-fd15-4d59-ac88-d01247fa4139-kube-api-access-dbm5q" (OuterVolumeSpecName: "kube-api-access-dbm5q") pod "f3a95e21-fd15-4d59-ac88-d01247fa4139" (UID: "f3a95e21-fd15-4d59-ac88-d01247fa4139"). InnerVolumeSpecName "kube-api-access-dbm5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:31:06.816113 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.816060    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3a95e21-fd15-4d59-ac88-d01247fa4139" (UID: "f3a95e21-fd15-4d59-ac88-d01247fa4139"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:31:06.914709 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.914630    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3a95e21-fd15-4d59-ac88-d01247fa4139-openshift-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:31:06.914709 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.914656    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3a95e21-fd15-4d59-ac88-d01247fa4139-maas-api-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:31:06.914709 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.914667    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbm5q\" (UniqueName: \"kubernetes.io/projected/f3a95e21-fd15-4d59-ac88-d01247fa4139-kube-api-access-dbm5q\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:31:06.917717 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.917690    2580 generic.go:356] "Generic (PLEG): container finished" podID="f3a95e21-fd15-4d59-ac88-d01247fa4139" containerID="e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682" exitCode=0
Jul 14 01:31:06.917860 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.917751    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"
Jul 14 01:31:06.917860 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.917762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b" event={"ID":"f3a95e21-fd15-4d59-ac88-d01247fa4139","Type":"ContainerDied","Data":"e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682"}
Jul 14 01:31:06.917860 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.917803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b" event={"ID":"f3a95e21-fd15-4d59-ac88-d01247fa4139","Type":"ContainerDied","Data":"ee190d92cf4e486ffdc58a773cf58e85426cbe0949db8cce49e5562b704e8a38"}
Jul 14 01:31:06.917860 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.917824    2580 scope.go:117] "RemoveContainer" containerID="e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682"
Jul 14 01:31:06.934775 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.934757    2580 scope.go:117] "RemoveContainer" containerID="e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682"
Jul 14 01:31:06.935034 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:31:06.935014    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682\": container with ID starting with e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682 not found: ID does not exist" containerID="e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682"
Jul 14 01:31:06.935096 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.935040    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682"} err="failed to get container status \"e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682\": rpc error: code = NotFound desc = could not find container \"e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682\": container with ID starting with e2abb3471498873b00e445866764321e9b5333d16a0d2af7459cc943a3c48682 not found: ID does not exist"
Jul 14 01:31:06.944051 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.944027    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"]
Jul 14 01:31:06.949239 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:06.949219    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7149234b-686c77dc78-msx2b"]
Jul 14 01:31:07.642561 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:07.642501    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a95e21-fd15-4d59-ac88-d01247fa4139" path="/var/lib/kubelet/pods/f3a95e21-fd15-4d59-ac88-d01247fa4139/volumes"
Jul 14 01:31:13.156012 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.155971    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"]
Jul 14 01:31:13.156620 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.156592    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a95e21-fd15-4d59-ac88-d01247fa4139" containerName="maas-api"
Jul 14 01:31:13.156620 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.156615    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a95e21-fd15-4d59-ac88-d01247fa4139" containerName="maas-api"
Jul 14 01:31:13.156752 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.156646    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5b0626d-8472-4371-881d-5d04faf65463" containerName="cleanup"
Jul 14 01:31:13.156752 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.156656    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5b0626d-8472-4371-881d-5d04faf65463" containerName="cleanup"
Jul 14 01:31:13.156846 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.156754    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5b0626d-8472-4371-881d-5d04faf65463" containerName="cleanup"
Jul 14 01:31:13.156846 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.156766    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a95e21-fd15-4d59-ac88-d01247fa4139" containerName="maas-api"
Jul 14 01:31:13.160012 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.159985    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.162024 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.161992    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0d59bf49-serving-cert\""
Jul 14 01:31:13.173078 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.173050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"]
Jul 14 01:31:13.279916 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.279869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-openshift-service-ca\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.279916 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.279916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq4x9\" (UniqueName: \"kubernetes.io/projected/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-kube-api-access-kq4x9\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.280164 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.279981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-maas-api-tls\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.380914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.380875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-openshift-service-ca\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.380914 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.380919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq4x9\" (UniqueName: \"kubernetes.io/projected/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-kube-api-access-kq4x9\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.381195 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.380947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-maas-api-tls\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.381693 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.381666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-openshift-service-ca\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.383479 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.383461    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-maas-api-tls\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.387872 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.387854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq4x9\" (UniqueName: \"kubernetes.io/projected/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-kube-api-access-kq4x9\") pod \"maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.476379 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.476284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:13.617135 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.617105    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"]
Jul 14 01:31:13.620101 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:31:13.620059    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f746ca_bf6f_4e1a_b387_ef8e246bcfa6.slice/crio-53285a9edd0b36e4fb046e875a67b6adab0b11f3fe8c626ce07e4ad0e81f847b WatchSource:0}: Error finding container 53285a9edd0b36e4fb046e875a67b6adab0b11f3fe8c626ce07e4ad0e81f847b: Status 404 returned error can't find the container with id 53285a9edd0b36e4fb046e875a67b6adab0b11f3fe8c626ce07e4ad0e81f847b
Jul 14 01:31:13.955369 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:13.955335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd" event={"ID":"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6","Type":"ContainerStarted","Data":"53285a9edd0b36e4fb046e875a67b6adab0b11f3fe8c626ce07e4ad0e81f847b"}
Jul 14 01:31:14.962515 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:14.962470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd" event={"ID":"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6","Type":"ContainerStarted","Data":"b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650"}
Jul 14 01:31:14.962936 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:14.962566    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:31:14.989446 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:14.989387    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd" podStartSLOduration=1.545414286 podStartE2EDuration="1.989366671s" totalImagesPullingTime="443.952385ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:31:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:31:14.988013983 +0000 UTC m=+1865.882864622" watchObservedRunningTime="2026-07-14 01:31:14.989366671 +0000 UTC m=+1865.884217310"
Jul 14 01:31:20.973354 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:31:20.973322    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:32:18.802506 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:18.802471    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"]
Jul 14 01:32:18.803177 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:18.802749    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd" podUID="86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" containerName="maas-api" containerID="cri-o://b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650" gracePeriod=30
Jul 14 01:32:19.050202 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.050176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:32:19.112797 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.112713    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-openshift-service-ca\") pod \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") "
Jul 14 01:32:19.112946 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.112807    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq4x9\" (UniqueName: \"kubernetes.io/projected/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-kube-api-access-kq4x9\") pod \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") "
Jul 14 01:32:19.112992 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.112956    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-maas-api-tls\") pod \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\" (UID: \"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6\") "
Jul 14 01:32:19.113087 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.113067    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" (UID: "86f746ca-bf6f-4e1a-b387-ef8e246bcfa6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:32:19.113286 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.113272    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-openshift-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:32:19.115022 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.114992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-kube-api-access-kq4x9" (OuterVolumeSpecName: "kube-api-access-kq4x9") pod "86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" (UID: "86f746ca-bf6f-4e1a-b387-ef8e246bcfa6"). InnerVolumeSpecName "kube-api-access-kq4x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:32:19.115143 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.115078    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" (UID: "86f746ca-bf6f-4e1a-b387-ef8e246bcfa6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:32:19.214427 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.214385    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq4x9\" (UniqueName: \"kubernetes.io/projected/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-kube-api-access-kq4x9\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:32:19.214427 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.214424    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6-maas-api-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:32:19.295975 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.295938    2580 generic.go:356] "Generic (PLEG): container finished" podID="86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" containerID="b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650" exitCode=0
Jul 14 01:32:19.296154 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.295998    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"
Jul 14 01:32:19.296154 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.296004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd" event={"ID":"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6","Type":"ContainerDied","Data":"b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650"}
Jul 14 01:32:19.296154 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.296062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd" event={"ID":"86f746ca-bf6f-4e1a-b387-ef8e246bcfa6","Type":"ContainerDied","Data":"53285a9edd0b36e4fb046e875a67b6adab0b11f3fe8c626ce07e4ad0e81f847b"}
Jul 14 01:32:19.296154 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.296086    2580 scope.go:117] "RemoveContainer" containerID="b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650"
Jul 14 01:32:19.312877 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.312856    2580 scope.go:117] "RemoveContainer" containerID="b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650"
Jul 14 01:32:19.313172 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:32:19.313155    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650\": container with ID starting with b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650 not found: ID does not exist" containerID="b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650"
Jul 14 01:32:19.313207 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.313180    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650"} err="failed to get container status \"b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650\": rpc error: code = NotFound desc = could not find container \"b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650\": container with ID starting with b231fefea6f717aa8fe69f6f976bc1676042749d80dbc37ef1d509e60462c650 not found: ID does not exist"
Jul 14 01:32:19.320850 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.320819    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"]
Jul 14 01:32:19.324556 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.324505    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0d59bf49-5bd75b699b-xvcpd"]
Jul 14 01:32:19.643953 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:19.643922    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" path="/var/lib/kubelet/pods/86f746ca-bf6f-4e1a-b387-ef8e246bcfa6/volumes"
Jul 14 01:32:35.815365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.815330    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"]
Jul 14 01:32:35.815925 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.815905    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" containerName="maas-api"
Jul 14 01:32:35.816011 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.815927    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" containerName="maas-api"
Jul 14 01:32:35.816064 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.816054    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="86f746ca-bf6f-4e1a-b387-ef8e246bcfa6" containerName="maas-api"
Jul 14 01:32:35.820773 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.820754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:35.822753 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.822731    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0e23d941-serving-cert\""
Jul 14 01:32:35.827916 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.827892    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"]
Jul 14 01:32:35.877965 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.877938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-openshift-service-ca\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:35.878088 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.877981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv487\" (UniqueName: \"kubernetes.io/projected/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-kube-api-access-bv487\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:35.878088 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.878062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:35.978772 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.978746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-openshift-service-ca\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:35.978908 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.978784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv487\" (UniqueName: \"kubernetes.io/projected/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-kube-api-access-bv487\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:35.978908 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.978826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:35.979018 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:32:35.978925    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-serving-cert: secret "maas-api-e2e-derive-0e23d941-serving-cert" not found
Jul 14 01:32:35.979018 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:32:35.978980    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls podName:ec73c467-8df0-4fbe-9f8f-e5757fd4e792 nodeName:}" failed. No retries permitted until 2026-07-14 01:32:36.478961662 +0000 UTC m=+1947.373812279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls") pod "maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt" (UID: "ec73c467-8df0-4fbe-9f8f-e5757fd4e792") : secret "maas-api-e2e-derive-0e23d941-serving-cert" not found
Jul 14 01:32:35.979399 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.979378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-openshift-service-ca\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:35.986472 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:35.986449    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv487\" (UniqueName: \"kubernetes.io/projected/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-kube-api-access-bv487\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:36.483422 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:36.483389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:36.483617 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:32:36.483549    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-serving-cert: secret "maas-api-e2e-derive-0e23d941-serving-cert" not found
Jul 14 01:32:36.483617 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:32:36.483612    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls podName:ec73c467-8df0-4fbe-9f8f-e5757fd4e792 nodeName:}" failed. No retries permitted until 2026-07-14 01:32:37.483597876 +0000 UTC m=+1948.378448496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls") pod "maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt" (UID: "ec73c467-8df0-4fbe-9f8f-e5757fd4e792") : secret "maas-api-e2e-derive-0e23d941-serving-cert" not found
Jul 14 01:32:37.493969 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:37.493935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:37.496421 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:37.496402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls\") pod \"maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:37.634422 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:37.634391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:37.765660 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:37.765633    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"]
Jul 14 01:32:37.768074 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:32:37.768047    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec73c467_8df0_4fbe_9f8f_e5757fd4e792.slice/crio-7c80a4fd2c82717e4b40c3e6d893f76601e067480acdafae992ca277bf112e3c WatchSource:0}: Error finding container 7c80a4fd2c82717e4b40c3e6d893f76601e067480acdafae992ca277bf112e3c: Status 404 returned error can't find the container with id 7c80a4fd2c82717e4b40c3e6d893f76601e067480acdafae992ca277bf112e3c
Jul 14 01:32:38.396775 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:38.396746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt" event={"ID":"ec73c467-8df0-4fbe-9f8f-e5757fd4e792","Type":"ContainerStarted","Data":"48d3cf1c238526172d134e721be770645f776de7013b88257506eaea815f5383"}
Jul 14 01:32:38.396944 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:38.396793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt" event={"ID":"ec73c467-8df0-4fbe-9f8f-e5757fd4e792","Type":"ContainerStarted","Data":"7c80a4fd2c82717e4b40c3e6d893f76601e067480acdafae992ca277bf112e3c"}
Jul 14 01:32:38.396944 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:38.396836    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:32:38.414733 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:38.414653    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt" podStartSLOduration=3.026173392 podStartE2EDuration="3.414636906s" totalImagesPullingTime="388.463514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:32:38.412231379 +0000 UTC m=+1949.307082020" watchObservedRunningTime="2026-07-14 01:32:38.414636906 +0000 UTC m=+1949.309487581"
Jul 14 01:32:44.406976 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:32:44.406942    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:33:50.602550 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.602496    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"]
Jul 14 01:33:50.603072 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.602762    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt" podUID="ec73c467-8df0-4fbe-9f8f-e5757fd4e792" containerName="maas-api" containerID="cri-o://48d3cf1c238526172d134e721be770645f776de7013b88257506eaea815f5383" gracePeriod=30
Jul 14 01:33:50.763683 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.763652    2580 generic.go:356] "Generic (PLEG): container finished" podID="ec73c467-8df0-4fbe-9f8f-e5757fd4e792" containerID="48d3cf1c238526172d134e721be770645f776de7013b88257506eaea815f5383" exitCode=0
Jul 14 01:33:50.763832 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.763690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt" event={"ID":"ec73c467-8df0-4fbe-9f8f-e5757fd4e792","Type":"ContainerDied","Data":"48d3cf1c238526172d134e721be770645f776de7013b88257506eaea815f5383"}
Jul 14 01:33:50.852672 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.852610    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:33:50.899706 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.899681    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-openshift-service-ca\") pod \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") "
Jul 14 01:33:50.899827 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.899746    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv487\" (UniqueName: \"kubernetes.io/projected/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-kube-api-access-bv487\") pod \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") "
Jul 14 01:33:50.899876 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.899851    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls\") pod \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\" (UID: \"ec73c467-8df0-4fbe-9f8f-e5757fd4e792\") "
Jul 14 01:33:50.900095 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.900058    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec73c467-8df0-4fbe-9f8f-e5757fd4e792" (UID: "ec73c467-8df0-4fbe-9f8f-e5757fd4e792"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:33:50.900213 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.900136    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-openshift-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:33:50.901906 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.901886    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-kube-api-access-bv487" (OuterVolumeSpecName: "kube-api-access-bv487") pod "ec73c467-8df0-4fbe-9f8f-e5757fd4e792" (UID: "ec73c467-8df0-4fbe-9f8f-e5757fd4e792"). InnerVolumeSpecName "kube-api-access-bv487". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:33:50.902005 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:50.901930    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec73c467-8df0-4fbe-9f8f-e5757fd4e792" (UID: "ec73c467-8df0-4fbe-9f8f-e5757fd4e792"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:33:51.001225 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:51.001202    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv487\" (UniqueName: \"kubernetes.io/projected/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-kube-api-access-bv487\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:33:51.001225 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:51.001224    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec73c467-8df0-4fbe-9f8f-e5757fd4e792-maas-api-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:33:51.769905 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:51.769862    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt" event={"ID":"ec73c467-8df0-4fbe-9f8f-e5757fd4e792","Type":"ContainerDied","Data":"7c80a4fd2c82717e4b40c3e6d893f76601e067480acdafae992ca277bf112e3c"}
Jul 14 01:33:51.769905 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:51.769892    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"
Jul 14 01:33:51.770402 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:51.769938    2580 scope.go:117] "RemoveContainer" containerID="48d3cf1c238526172d134e721be770645f776de7013b88257506eaea815f5383"
Jul 14 01:33:51.790610 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:51.790561    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"]
Jul 14 01:33:51.792177 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:51.792154    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e23d941-74c8487b6-qjrpt"]
Jul 14 01:33:53.642916 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:33:53.642879    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec73c467-8df0-4fbe-9f8f-e5757fd4e792" path="/var/lib/kubelet/pods/ec73c467-8df0-4fbe-9f8f-e5757fd4e792/volumes"
Jul 14 01:35:44.673340 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.673308    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"]
Jul 14 01:35:44.674049 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.673721    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec73c467-8df0-4fbe-9f8f-e5757fd4e792" containerName="maas-api"
Jul 14 01:35:44.674049 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.673733    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec73c467-8df0-4fbe-9f8f-e5757fd4e792" containerName="maas-api"
Jul 14 01:35:44.674049 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.673853    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec73c467-8df0-4fbe-9f8f-e5757fd4e792" containerName="maas-api"
Jul 14 01:35:44.677155 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.677134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:44.679058 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.679039    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-db5a4a31-serving-cert\""
Jul 14 01:35:44.686958 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.686934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"]
Jul 14 01:35:44.772728 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.772697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-openshift-service-ca\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:44.772893 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.772737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2587n\" (UniqueName: \"kubernetes.io/projected/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-kube-api-access-2587n\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:44.772893 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.772769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:44.873458 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.873422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-openshift-service-ca\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:44.873458 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.873458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2587n\" (UniqueName: \"kubernetes.io/projected/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-kube-api-access-2587n\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:44.873736 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.873491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:44.873736 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:35:44.873624    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-serving-cert: secret "maas-api-e2e-mt-db5a4a31-serving-cert" not found
Jul 14 01:35:44.873736 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:35:44.873694    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls podName:f1c0affc-8d5e-4fe2-a91a-e195d7561bbc nodeName:}" failed. No retries permitted until 2026-07-14 01:35:45.373674665 +0000 UTC m=+2136.268525299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls") pod "maas-api-e2e-mt-db5a4a31-68847695f9-vc486" (UID: "f1c0affc-8d5e-4fe2-a91a-e195d7561bbc") : secret "maas-api-e2e-mt-db5a4a31-serving-cert" not found
Jul 14 01:35:44.874047 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.874030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-openshift-service-ca\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:44.891907 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:44.891885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2587n\" (UniqueName: \"kubernetes.io/projected/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-kube-api-access-2587n\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:45.377088 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:45.377051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:45.379717 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:45.379688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls\") pod \"maas-api-e2e-mt-db5a4a31-68847695f9-vc486\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:45.589547 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:45.589492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:45.733073 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:45.733049    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"]
Jul 14 01:35:45.737664 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:45.737637    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 01:35:46.317773 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:46.317741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486" event={"ID":"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc","Type":"ContainerStarted","Data":"c2849c5d942438b5b497bd6ce659d938f37ed29d0576b6aa08ecb830f00e25b9"}
Jul 14 01:35:47.324599 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:47.324566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486" event={"ID":"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc","Type":"ContainerStarted","Data":"d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777"}
Jul 14 01:35:47.324993 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:47.324734    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:35:47.341274 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:47.341227    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486" podStartSLOduration=2.814657672 podStartE2EDuration="3.341209478s" totalImagesPullingTime="526.551806ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:35:47.340111471 +0000 UTC m=+2138.234962111" watchObservedRunningTime="2026-07-14 01:35:47.341209478 +0000 UTC m=+2138.236060117"
Jul 14 01:35:53.334873 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:35:53.334845    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:36:06.679968 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:06.679875    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"]
Jul 14 01:36:06.680452 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:06.680088    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486" podUID="f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" containerName="maas-api" containerID="cri-o://d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777" gracePeriod=30
Jul 14 01:36:06.934667 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:06.934610    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:36:07.062863 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.062834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-openshift-service-ca\") pod \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") "
Jul 14 01:36:07.063049 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.062868    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls\") pod \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") "
Jul 14 01:36:07.063049 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.062890    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2587n\" (UniqueName: \"kubernetes.io/projected/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-kube-api-access-2587n\") pod \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\" (UID: \"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc\") "
Jul 14 01:36:07.063228 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.063204    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" (UID: "f1c0affc-8d5e-4fe2-a91a-e195d7561bbc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:36:07.065082 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.065056    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-kube-api-access-2587n" (OuterVolumeSpecName: "kube-api-access-2587n") pod "f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" (UID: "f1c0affc-8d5e-4fe2-a91a-e195d7561bbc"). InnerVolumeSpecName "kube-api-access-2587n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:36:07.065220 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.065124    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" (UID: "f1c0affc-8d5e-4fe2-a91a-e195d7561bbc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 01:36:07.164281 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.164254    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-openshift-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:36:07.164281 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.164278    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-maas-api-tls\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:36:07.164281 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.164287    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2587n\" (UniqueName: \"kubernetes.io/projected/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc-kube-api-access-2587n\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:36:07.428778 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.428743    2580 generic.go:356] "Generic (PLEG): container finished" podID="f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" containerID="d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777" exitCode=0
Jul 14 01:36:07.428945 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.428791    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"
Jul 14 01:36:07.428945 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.428833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486" event={"ID":"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc","Type":"ContainerDied","Data":"d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777"}
Jul 14 01:36:07.428945 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.428893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486" event={"ID":"f1c0affc-8d5e-4fe2-a91a-e195d7561bbc","Type":"ContainerDied","Data":"c2849c5d942438b5b497bd6ce659d938f37ed29d0576b6aa08ecb830f00e25b9"}
Jul 14 01:36:07.428945 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.428918    2580 scope.go:117] "RemoveContainer" containerID="d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777"
Jul 14 01:36:07.438718 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.438703    2580 scope.go:117] "RemoveContainer" containerID="d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777"
Jul 14 01:36:07.438944 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:36:07.438925    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777\": container with ID starting with d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777 not found: ID does not exist" containerID="d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777"
Jul 14 01:36:07.438991 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.438954    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777"} err="failed to get container status \"d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777\": rpc error: code = NotFound desc = could not find container \"d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777\": container with ID starting with d5a36a41b94daf4d80f9b1459340595a98e7d7ad6ea86e923ca977bb51a1f777 not found: ID does not exist"
Jul 14 01:36:07.447671 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.447635    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"]
Jul 14 01:36:07.449933 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.449913    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-db5a4a31-68847695f9-vc486"]
Jul 14 01:36:07.643368 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:36:07.643336    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" path="/var/lib/kubelet/pods/f1c0affc-8d5e-4fe2-a91a-e195d7561bbc/volumes"
Jul 14 01:37:37.827986 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:37.827907    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"]
Jul 14 01:37:37.828469 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:37.828319    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" containerName="maas-api"
Jul 14 01:37:37.828469 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:37.828339    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" containerName="maas-api"
Jul 14 01:37:37.828469 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:37.828440    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1c0affc-8d5e-4fe2-a91a-e195d7561bbc" containerName="maas-api"
Jul 14 01:37:37.831814 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:37.831798    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"
Jul 14 01:37:37.833703 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:37.833681    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:37:37.837120 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:37.837099    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"]
Jul 14 01:37:37.907023 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:37.906999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf44989-b22f-4b03-90c2-6ac6edb88dfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt\" (UID: \"1bf44989-b22f-4b03-90c2-6ac6edb88dfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"
Jul 14 01:37:38.008418 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:38.008394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf44989-b22f-4b03-90c2-6ac6edb88dfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt\" (UID: \"1bf44989-b22f-4b03-90c2-6ac6edb88dfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"
Jul 14 01:37:38.008984 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:38.008965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf44989-b22f-4b03-90c2-6ac6edb88dfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt\" (UID: \"1bf44989-b22f-4b03-90c2-6ac6edb88dfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"
Jul 14 01:37:38.143736 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:38.143675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"
Jul 14 01:37:38.279408 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:38.279380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"]
Jul 14 01:37:38.281514 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:37:38.281488    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf44989_b22f_4b03_90c2_6ac6edb88dfc.slice/crio-7f3ac40083edbcfe08237eb4745b7e434480b864627428744c2ae9c0488f1454 WatchSource:0}: Error finding container 7f3ac40083edbcfe08237eb4745b7e434480b864627428744c2ae9c0488f1454: Status 404 returned error can't find the container with id 7f3ac40083edbcfe08237eb4745b7e434480b864627428744c2ae9c0488f1454
Jul 14 01:37:38.845069 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:38.845035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt" event={"ID":"1bf44989-b22f-4b03-90c2-6ac6edb88dfc","Type":"ContainerStarted","Data":"2141c006a6cc7f042b70bffc46d853b6aaf3fce076ba8c2b712404fc0c6bf708"}
Jul 14 01:37:38.845422 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:38.845090    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt" event={"ID":"1bf44989-b22f-4b03-90c2-6ac6edb88dfc","Type":"ContainerStarted","Data":"7f3ac40083edbcfe08237eb4745b7e434480b864627428744c2ae9c0488f1454"}
Jul 14 01:37:38.859985 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:38.859937    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt" podStartSLOduration=1.8599218849999999 podStartE2EDuration="1.859921885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:37:38.858631975 +0000 UTC m=+2249.753482618" watchObservedRunningTime="2026-07-14 01:37:38.859921885 +0000 UTC m=+2249.754772522"
Jul 14 01:37:39.851560 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:39.851509    2580 generic.go:356] "Generic (PLEG): container finished" podID="1bf44989-b22f-4b03-90c2-6ac6edb88dfc" containerID="2141c006a6cc7f042b70bffc46d853b6aaf3fce076ba8c2b712404fc0c6bf708" exitCode=0
Jul 14 01:37:39.851936 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:39.851558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt" event={"ID":"1bf44989-b22f-4b03-90c2-6ac6edb88dfc","Type":"ContainerDied","Data":"2141c006a6cc7f042b70bffc46d853b6aaf3fce076ba8c2b712404fc0c6bf708"}
Jul 14 01:37:40.976610 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:40.976585    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"
Jul 14 01:37:41.035127 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:41.035096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf44989-b22f-4b03-90c2-6ac6edb88dfc-maas-api-service-ca\") pod \"1bf44989-b22f-4b03-90c2-6ac6edb88dfc\" (UID: \"1bf44989-b22f-4b03-90c2-6ac6edb88dfc\") "
Jul 14 01:37:41.035433 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:41.035413    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf44989-b22f-4b03-90c2-6ac6edb88dfc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1bf44989-b22f-4b03-90c2-6ac6edb88dfc" (UID: "1bf44989-b22f-4b03-90c2-6ac6edb88dfc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:37:41.136759 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:41.136692    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bf44989-b22f-4b03-90c2-6ac6edb88dfc-maas-api-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:37:41.862186 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:41.862160    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt"
Jul 14 01:37:41.862345 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:41.862184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt" event={"ID":"1bf44989-b22f-4b03-90c2-6ac6edb88dfc","Type":"ContainerDied","Data":"7f3ac40083edbcfe08237eb4745b7e434480b864627428744c2ae9c0488f1454"}
Jul 14 01:37:41.862345 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:37:41.862227    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f3ac40083edbcfe08237eb4745b7e434480b864627428744c2ae9c0488f1454"
Jul 14 01:41:29.611956 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.611917    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"]
Jul 14 01:41:29.612563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.612294    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bf44989-b22f-4b03-90c2-6ac6edb88dfc" containerName="revoke-keys"
Jul 14 01:41:29.612563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.612304    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bf44989-b22f-4b03-90c2-6ac6edb88dfc" containerName="revoke-keys"
Jul 14 01:41:29.612563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.612381    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bf44989-b22f-4b03-90c2-6ac6edb88dfc" containerName="revoke-keys"
Jul 14 01:41:29.615674 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.615655    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"
Jul 14 01:41:29.617449 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.617430    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:41:29.620975 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.620956    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"]
Jul 14 01:41:29.771853 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.771823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5887c777-be21-474e-a52b-1795e2785ef5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6\" (UID: \"5887c777-be21-474e-a52b-1795e2785ef5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"
Jul 14 01:41:29.872641 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.872565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5887c777-be21-474e-a52b-1795e2785ef5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6\" (UID: \"5887c777-be21-474e-a52b-1795e2785ef5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"
Jul 14 01:41:29.873170 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.873149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5887c777-be21-474e-a52b-1795e2785ef5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6\" (UID: \"5887c777-be21-474e-a52b-1795e2785ef5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"
Jul 14 01:41:29.926882 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:29.926861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"
Jul 14 01:41:30.055196 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:30.055171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"]
Jul 14 01:41:30.057630 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:41:30.057598    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5887c777_be21_474e_a52b_1795e2785ef5.slice/crio-be4f7e3338a5b9b9115087c6314b352cf38f895d99b5b38b860a7642f7d7f523 WatchSource:0}: Error finding container be4f7e3338a5b9b9115087c6314b352cf38f895d99b5b38b860a7642f7d7f523: Status 404 returned error can't find the container with id be4f7e3338a5b9b9115087c6314b352cf38f895d99b5b38b860a7642f7d7f523
Jul 14 01:41:30.933204 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:30.933174    2580 generic.go:356] "Generic (PLEG): container finished" podID="5887c777-be21-474e-a52b-1795e2785ef5" containerID="9d33ff1ac2dddf51dba2d503eed57c3149c8e0b3b5031304b208ddf78f4b71f4" exitCode=0
Jul 14 01:41:30.933563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:30.933245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6" event={"ID":"5887c777-be21-474e-a52b-1795e2785ef5","Type":"ContainerDied","Data":"9d33ff1ac2dddf51dba2d503eed57c3149c8e0b3b5031304b208ddf78f4b71f4"}
Jul 14 01:41:30.933563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:30.933285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6" event={"ID":"5887c777-be21-474e-a52b-1795e2785ef5","Type":"ContainerStarted","Data":"be4f7e3338a5b9b9115087c6314b352cf38f895d99b5b38b860a7642f7d7f523"}
Jul 14 01:41:32.062345 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:32.062320    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"
Jul 14 01:41:32.192067 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:32.192010    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5887c777-be21-474e-a52b-1795e2785ef5-maas-api-service-ca\") pod \"5887c777-be21-474e-a52b-1795e2785ef5\" (UID: \"5887c777-be21-474e-a52b-1795e2785ef5\") "
Jul 14 01:41:32.192327 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:32.192308    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5887c777-be21-474e-a52b-1795e2785ef5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5887c777-be21-474e-a52b-1795e2785ef5" (UID: "5887c777-be21-474e-a52b-1795e2785ef5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:41:32.293058 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:32.293032    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5887c777-be21-474e-a52b-1795e2785ef5-maas-api-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:41:32.943395 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:32.943364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6" event={"ID":"5887c777-be21-474e-a52b-1795e2785ef5","Type":"ContainerDied","Data":"be4f7e3338a5b9b9115087c6314b352cf38f895d99b5b38b860a7642f7d7f523"}
Jul 14 01:41:32.943395 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:32.943394    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6"
Jul 14 01:41:32.943629 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:32.943410    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be4f7e3338a5b9b9115087c6314b352cf38f895d99b5b38b860a7642f7d7f523"
Jul 14 01:41:55.738400 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.738372    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"]
Jul 14 01:41:55.738852 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.738737    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5887c777-be21-474e-a52b-1795e2785ef5" containerName="revoke-keys"
Jul 14 01:41:55.738852 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.738748    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5887c777-be21-474e-a52b-1795e2785ef5" containerName="revoke-keys"
Jul 14 01:41:55.738852 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.738819    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5887c777-be21-474e-a52b-1795e2785ef5" containerName="revoke-keys"
Jul 14 01:41:55.741603 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.741588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"
Jul 14 01:41:55.743702 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.743670    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:41:55.747765 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.747745    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"]
Jul 14 01:41:55.779223 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.779197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2b2514-5f43-42fd-85b8-cedca21f2922-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm\" (UID: \"6e2b2514-5f43-42fd-85b8-cedca21f2922\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"
Jul 14 01:41:55.880126 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.880095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2b2514-5f43-42fd-85b8-cedca21f2922-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm\" (UID: \"6e2b2514-5f43-42fd-85b8-cedca21f2922\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"
Jul 14 01:41:55.880739 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:55.880720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2b2514-5f43-42fd-85b8-cedca21f2922-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm\" (UID: \"6e2b2514-5f43-42fd-85b8-cedca21f2922\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"
Jul 14 01:41:56.053660 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:56.053616    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"
Jul 14 01:41:56.188429 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:56.188406    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"]
Jul 14 01:41:56.190936 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:41:56.190907    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2b2514_5f43_42fd_85b8_cedca21f2922.slice/crio-28aa81a833d40e191e99670bb169bd120de0df97033929f0523000674f4598e2 WatchSource:0}: Error finding container 28aa81a833d40e191e99670bb169bd120de0df97033929f0523000674f4598e2: Status 404 returned error can't find the container with id 28aa81a833d40e191e99670bb169bd120de0df97033929f0523000674f4598e2
Jul 14 01:41:57.057945 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:57.057866    2580 generic.go:356] "Generic (PLEG): container finished" podID="6e2b2514-5f43-42fd-85b8-cedca21f2922" containerID="9c255e467a6f149948a44155d624f450adec25a938399e12fb2e6eb765385e7a" exitCode=0
Jul 14 01:41:57.057945 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:57.057912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm" event={"ID":"6e2b2514-5f43-42fd-85b8-cedca21f2922","Type":"ContainerDied","Data":"9c255e467a6f149948a44155d624f450adec25a938399e12fb2e6eb765385e7a"}
Jul 14 01:41:57.058330 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:57.057964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm" event={"ID":"6e2b2514-5f43-42fd-85b8-cedca21f2922","Type":"ContainerStarted","Data":"28aa81a833d40e191e99670bb169bd120de0df97033929f0523000674f4598e2"}
Jul 14 01:41:58.206254 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:58.206228    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"
Jul 14 01:41:58.301791 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:58.301764    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2b2514-5f43-42fd-85b8-cedca21f2922-maas-api-service-ca\") pod \"6e2b2514-5f43-42fd-85b8-cedca21f2922\" (UID: \"6e2b2514-5f43-42fd-85b8-cedca21f2922\") "
Jul 14 01:41:58.302106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:58.302083    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e2b2514-5f43-42fd-85b8-cedca21f2922-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e2b2514-5f43-42fd-85b8-cedca21f2922" (UID: "6e2b2514-5f43-42fd-85b8-cedca21f2922"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:41:58.402302 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:58.402242    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2b2514-5f43-42fd-85b8-cedca21f2922-maas-api-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:41:59.068538 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:59.068496    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm"
Jul 14 01:41:59.068723 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:59.068495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm" event={"ID":"6e2b2514-5f43-42fd-85b8-cedca21f2922","Type":"ContainerDied","Data":"28aa81a833d40e191e99670bb169bd120de0df97033929f0523000674f4598e2"}
Jul 14 01:41:59.068723 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:41:59.068637    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28aa81a833d40e191e99670bb169bd120de0df97033929f0523000674f4598e2"
Jul 14 01:42:46.588539 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.588495    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"]
Jul 14 01:42:46.589120 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.589066    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e2b2514-5f43-42fd-85b8-cedca21f2922" containerName="revoke-keys"
Jul 14 01:42:46.589120 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.589086    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e2b2514-5f43-42fd-85b8-cedca21f2922" containerName="revoke-keys"
Jul 14 01:42:46.589246 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.589184    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e2b2514-5f43-42fd-85b8-cedca21f2922" containerName="revoke-keys"
Jul 14 01:42:46.592319 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.592298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"
Jul 14 01:42:46.594055 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.594036    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:42:46.597328 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.597305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"]
Jul 14 01:42:46.689823 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.689792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b50e27-cc85-4237-9905-dcf5aaa8aae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-597948-m4rvh\" (UID: \"90b50e27-cc85-4237-9905-dcf5aaa8aae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"
Jul 14 01:42:46.790598 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.790565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b50e27-cc85-4237-9905-dcf5aaa8aae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-597948-m4rvh\" (UID: \"90b50e27-cc85-4237-9905-dcf5aaa8aae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"
Jul 14 01:42:46.791162 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.791144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b50e27-cc85-4237-9905-dcf5aaa8aae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-597948-m4rvh\" (UID: \"90b50e27-cc85-4237-9905-dcf5aaa8aae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"
Jul 14 01:42:46.904233 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:46.904161    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"
Jul 14 01:42:47.037601 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:47.037572    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"]
Jul 14 01:42:47.044609 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:42:47.044583    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b50e27_cc85_4237_9905_dcf5aaa8aae5.slice/crio-9e97ff5db7e3581009a287d442956ccff7faf934bb946e3dca7a081fbd4d7edf WatchSource:0}: Error finding container 9e97ff5db7e3581009a287d442956ccff7faf934bb946e3dca7a081fbd4d7edf: Status 404 returned error can't find the container with id 9e97ff5db7e3581009a287d442956ccff7faf934bb946e3dca7a081fbd4d7edf
Jul 14 01:42:47.313530 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:47.313480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh" event={"ID":"90b50e27-cc85-4237-9905-dcf5aaa8aae5","Type":"ContainerStarted","Data":"30e35c07cfcfd99eb452740da339bc6767d7367c8d1b716f1df60ae62fc8ab06"}
Jul 14 01:42:47.313650 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:47.313561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh" event={"ID":"90b50e27-cc85-4237-9905-dcf5aaa8aae5","Type":"ContainerStarted","Data":"9e97ff5db7e3581009a287d442956ccff7faf934bb946e3dca7a081fbd4d7edf"}
Jul 14 01:42:47.329116 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:47.329064    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh" podStartSLOduration=1.329046232 podStartE2EDuration="1.329046232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:42:47.327957287 +0000 UTC m=+2558.222807925" watchObservedRunningTime="2026-07-14 01:42:47.329046232 +0000 UTC m=+2558.223896869"
Jul 14 01:42:48.319935 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:48.319896    2580 generic.go:356] "Generic (PLEG): container finished" podID="90b50e27-cc85-4237-9905-dcf5aaa8aae5" containerID="30e35c07cfcfd99eb452740da339bc6767d7367c8d1b716f1df60ae62fc8ab06" exitCode=0
Jul 14 01:42:48.319935 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:48.319929    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh" event={"ID":"90b50e27-cc85-4237-9905-dcf5aaa8aae5","Type":"ContainerDied","Data":"30e35c07cfcfd99eb452740da339bc6767d7367c8d1b716f1df60ae62fc8ab06"}
Jul 14 01:42:49.454426 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:49.454402    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"
Jul 14 01:42:49.514646 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:49.514618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b50e27-cc85-4237-9905-dcf5aaa8aae5-maas-api-service-ca\") pod \"90b50e27-cc85-4237-9905-dcf5aaa8aae5\" (UID: \"90b50e27-cc85-4237-9905-dcf5aaa8aae5\") "
Jul 14 01:42:49.514927 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:49.514907    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90b50e27-cc85-4237-9905-dcf5aaa8aae5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90b50e27-cc85-4237-9905-dcf5aaa8aae5" (UID: "90b50e27-cc85-4237-9905-dcf5aaa8aae5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:42:49.615625 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:49.615557    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b50e27-cc85-4237-9905-dcf5aaa8aae5-maas-api-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:42:50.332779 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:50.332756    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh"
Jul 14 01:42:50.332779 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:50.332737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-597948-m4rvh" event={"ID":"90b50e27-cc85-4237-9905-dcf5aaa8aae5","Type":"ContainerDied","Data":"9e97ff5db7e3581009a287d442956ccff7faf934bb946e3dca7a081fbd4d7edf"}
Jul 14 01:42:50.332985 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:42:50.332836    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e97ff5db7e3581009a287d442956ccff7faf934bb946e3dca7a081fbd4d7edf"
Jul 14 01:43:12.705754 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.705711    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"]
Jul 14 01:43:12.706219 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.706104    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b50e27-cc85-4237-9905-dcf5aaa8aae5" containerName="revoke-keys"
Jul 14 01:43:12.706219 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.706116    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b50e27-cc85-4237-9905-dcf5aaa8aae5" containerName="revoke-keys"
Jul 14 01:43:12.706219 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.706190    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b50e27-cc85-4237-9905-dcf5aaa8aae5" containerName="revoke-keys"
Jul 14 01:43:12.709542 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.709495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"
Jul 14 01:43:12.711436 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.711409    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:43:12.715068 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.715038    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"]
Jul 14 01:43:12.827317 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.827282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50aed537-dbdc-494a-8171-59abcd449038-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2\" (UID: \"50aed537-dbdc-494a-8171-59abcd449038\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"
Jul 14 01:43:12.928038 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.928010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50aed537-dbdc-494a-8171-59abcd449038-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2\" (UID: \"50aed537-dbdc-494a-8171-59abcd449038\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"
Jul 14 01:43:12.928645 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:12.928626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50aed537-dbdc-494a-8171-59abcd449038-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2\" (UID: \"50aed537-dbdc-494a-8171-59abcd449038\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"
Jul 14 01:43:13.021859 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:13.021840    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"
Jul 14 01:43:13.150829 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:13.150799    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"]
Jul 14 01:43:13.154808 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:43:13.154768    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50aed537_dbdc_494a_8171_59abcd449038.slice/crio-865f7ded27df4e8ae1b706bbc07255d0988bd4c88ee71cdd428be2f85818105c WatchSource:0}: Error finding container 865f7ded27df4e8ae1b706bbc07255d0988bd4c88ee71cdd428be2f85818105c: Status 404 returned error can't find the container with id 865f7ded27df4e8ae1b706bbc07255d0988bd4c88ee71cdd428be2f85818105c
Jul 14 01:43:13.446829 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:13.446798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2" event={"ID":"50aed537-dbdc-494a-8171-59abcd449038","Type":"ContainerStarted","Data":"41c473c679b7b5f220872b48ca51614df242cdcde9eb9a45f777bbd371458c0e"}
Jul 14 01:43:13.446950 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:13.446847    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2" event={"ID":"50aed537-dbdc-494a-8171-59abcd449038","Type":"ContainerStarted","Data":"865f7ded27df4e8ae1b706bbc07255d0988bd4c88ee71cdd428be2f85818105c"}
Jul 14 01:43:13.460967 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:13.460922    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2" podStartSLOduration=1.460906845 podStartE2EDuration="1.460906845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:43:13.46001472 +0000 UTC m=+2584.354865377" watchObservedRunningTime="2026-07-14 01:43:13.460906845 +0000 UTC m=+2584.355757483"
Jul 14 01:43:14.453203 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:14.453168    2580 generic.go:356] "Generic (PLEG): container finished" podID="50aed537-dbdc-494a-8171-59abcd449038" containerID="41c473c679b7b5f220872b48ca51614df242cdcde9eb9a45f777bbd371458c0e" exitCode=0
Jul 14 01:43:14.453602 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:14.453230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2" event={"ID":"50aed537-dbdc-494a-8171-59abcd449038","Type":"ContainerDied","Data":"41c473c679b7b5f220872b48ca51614df242cdcde9eb9a45f777bbd371458c0e"}
Jul 14 01:43:15.588924 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:15.588899    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"
Jul 14 01:43:15.756104 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:15.756077    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50aed537-dbdc-494a-8171-59abcd449038-maas-api-service-ca\") pod \"50aed537-dbdc-494a-8171-59abcd449038\" (UID: \"50aed537-dbdc-494a-8171-59abcd449038\") "
Jul 14 01:43:15.756424 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:15.756402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50aed537-dbdc-494a-8171-59abcd449038-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50aed537-dbdc-494a-8171-59abcd449038" (UID: "50aed537-dbdc-494a-8171-59abcd449038"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:43:15.857542 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:15.857499    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50aed537-dbdc-494a-8171-59abcd449038-maas-api-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:43:16.464498 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:16.464469    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2"
Jul 14 01:43:16.464498 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:16.464481    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2" event={"ID":"50aed537-dbdc-494a-8171-59abcd449038","Type":"ContainerDied","Data":"865f7ded27df4e8ae1b706bbc07255d0988bd4c88ee71cdd428be2f85818105c"}
Jul 14 01:43:16.464768 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:43:16.464558    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="865f7ded27df4e8ae1b706bbc07255d0988bd4c88ee71cdd428be2f85818105c"
Jul 14 01:45:00.132365 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.132275    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5"]
Jul 14 01:45:00.132856 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.132733    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50aed537-dbdc-494a-8171-59abcd449038" containerName="revoke-keys"
Jul 14 01:45:00.132856 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.132752    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="50aed537-dbdc-494a-8171-59abcd449038" containerName="revoke-keys"
Jul 14 01:45:00.132932 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.132880    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="50aed537-dbdc-494a-8171-59abcd449038" containerName="revoke-keys"
Jul 14 01:45:00.136005 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.135986    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"]
Jul 14 01:45:00.136103 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.136086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5"
Jul 14 01:45:00.137972 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.137955    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:45:00.139838 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.139822    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"]
Jul 14 01:45:00.139928 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.139914    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"
Jul 14 01:45:00.145447 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.144827    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"]
Jul 14 01:45:00.145447 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.144966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"
Jul 14 01:45:00.146725 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.146706    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"]
Jul 14 01:45:00.149389 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.149365    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5"]
Jul 14 01:45:00.150295 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.150276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5"
Jul 14 01:45:00.160350 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.160329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"
Jul 14 01:45:00.195865 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.195829    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"
Jul 14 01:45:00.392248 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.392223    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"]
Jul 14 01:45:00.394591 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:45:00.394559    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7930731_9e5f_4c93_8c71_3b867883eec5.slice/crio-1a22c90d46d135dbb4e821279b4734abcd6bfd572d0623689bfa6448ec3ca295 WatchSource:0}: Error finding container 1a22c90d46d135dbb4e821279b4734abcd6bfd572d0623689bfa6448ec3ca295: Status 404 returned error can't find the container with id 1a22c90d46d135dbb4e821279b4734abcd6bfd572d0623689bfa6448ec3ca295
Jul 14 01:45:00.532327 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.532303    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5"]
Jul 14 01:45:00.534298 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:45:00.534275    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e586fd_1f3f_4161_9d5d_103916ebd752.slice/crio-d18a9a51ce0a815d93597ddb43047315c42f6b0a592908663ebd948ecf9d9048 WatchSource:0}: Error finding container d18a9a51ce0a815d93597ddb43047315c42f6b0a592908663ebd948ecf9d9048: Status 404 returned error can't find the container with id d18a9a51ce0a815d93597ddb43047315c42f6b0a592908663ebd948ecf9d9048
Jul 14 01:45:00.565085 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.565063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"]
Jul 14 01:45:00.567235 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:45:00.567209    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1bcf81_bc9b_4547_b1e4_4954f4378a09.slice/crio-69b5c8f95e36a948e51fab07295c60fdeac5669bf1b95c3fa135a8da3459d7dd WatchSource:0}: Error finding container 69b5c8f95e36a948e51fab07295c60fdeac5669bf1b95c3fa135a8da3459d7dd: Status 404 returned error can't find the container with id 69b5c8f95e36a948e51fab07295c60fdeac5669bf1b95c3fa135a8da3459d7dd
Jul 14 01:45:00.965244 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.965151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5" event={"ID":"75e586fd-1f3f-4161-9d5d-103916ebd752","Type":"ContainerStarted","Data":"287d0480b82dd8990973d78ae11d1d155cd7b52b66ec2f45727fcc33be716b2a"}
Jul 14 01:45:00.965244 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.965212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5" event={"ID":"75e586fd-1f3f-4161-9d5d-103916ebd752","Type":"ContainerStarted","Data":"d18a9a51ce0a815d93597ddb43047315c42f6b0a592908663ebd948ecf9d9048"}
Jul 14 01:45:00.966728 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.966703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5" event={"ID":"9c1bcf81-bc9b-4547-b1e4-4954f4378a09","Type":"ContainerStarted","Data":"8952f984630bd7fc5a9f3729c734d5984d15eea03cceb1c0036336b2042024bb"}
Jul 14 01:45:00.966830 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.966758    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5" event={"ID":"9c1bcf81-bc9b-4547-b1e4-4954f4378a09","Type":"ContainerStarted","Data":"69b5c8f95e36a948e51fab07295c60fdeac5669bf1b95c3fa135a8da3459d7dd"}
Jul 14 01:45:00.968250 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.968227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk" event={"ID":"a7930731-9e5f-4c93-8c71-3b867883eec5","Type":"ContainerStarted","Data":"49c8e1c5703023bacb721e5aa46a15f73a0e51d873a83ec716b362d0706a0459"}
Jul 14 01:45:00.968355 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.968267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk" event={"ID":"a7930731-9e5f-4c93-8c71-3b867883eec5","Type":"ContainerStarted","Data":"1a22c90d46d135dbb4e821279b4734abcd6bfd572d0623689bfa6448ec3ca295"}
Jul 14 01:45:00.978952 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.978906    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5" podStartSLOduration=0.978891068 podStartE2EDuration="978.891068ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:45:00.977060065 +0000 UTC m=+2691.871910703" watchObservedRunningTime="2026-07-14 01:45:00.978891068 +0000 UTC m=+2691.873741705"
Jul 14 01:45:00.990313 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:00.990278    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5" podStartSLOduration=0.990264054 podStartE2EDuration="990.264054ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:45:00.989375777 +0000 UTC m=+2691.884226415" watchObservedRunningTime="2026-07-14 01:45:00.990264054 +0000 UTC m=+2691.885114693"
Jul 14 01:45:01.002480 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:01.002334    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk" podStartSLOduration=1.002317195 podStartE2EDuration="1.002317195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:45:01.002003226 +0000 UTC m=+2691.896853881" watchObservedRunningTime="2026-07-14 01:45:01.002317195 +0000 UTC m=+2691.897167835"
Jul 14 01:45:01.974972 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:01.974936    2580 generic.go:356] "Generic (PLEG): container finished" podID="9c1bcf81-bc9b-4547-b1e4-4954f4378a09" containerID="8952f984630bd7fc5a9f3729c734d5984d15eea03cceb1c0036336b2042024bb" exitCode=0
Jul 14 01:45:01.975343 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:01.975020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5" event={"ID":"9c1bcf81-bc9b-4547-b1e4-4954f4378a09","Type":"ContainerDied","Data":"8952f984630bd7fc5a9f3729c734d5984d15eea03cceb1c0036336b2042024bb"}
Jul 14 01:45:01.976449 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:01.976428    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7930731-9e5f-4c93-8c71-3b867883eec5" containerID="49c8e1c5703023bacb721e5aa46a15f73a0e51d873a83ec716b362d0706a0459" exitCode=0
Jul 14 01:45:01.976587 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:01.976500    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk" event={"ID":"a7930731-9e5f-4c93-8c71-3b867883eec5","Type":"ContainerDied","Data":"49c8e1c5703023bacb721e5aa46a15f73a0e51d873a83ec716b362d0706a0459"}
Jul 14 01:45:01.977960 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:01.977939    2580 generic.go:356] "Generic (PLEG): container finished" podID="75e586fd-1f3f-4161-9d5d-103916ebd752" containerID="287d0480b82dd8990973d78ae11d1d155cd7b52b66ec2f45727fcc33be716b2a" exitCode=0
Jul 14 01:45:01.978053 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:01.977971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5" event={"ID":"75e586fd-1f3f-4161-9d5d-103916ebd752","Type":"ContainerDied","Data":"287d0480b82dd8990973d78ae11d1d155cd7b52b66ec2f45727fcc33be716b2a"}
Jul 14 01:45:03.139404 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.139380    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"
Jul 14 01:45:03.142299 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.142282    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"
Jul 14 01:45:03.178201 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.178181    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5"
Jul 14 01:45:03.988936 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.988862    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5"
Jul 14 01:45:03.989087 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.988869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-wmfv5" event={"ID":"75e586fd-1f3f-4161-9d5d-103916ebd752","Type":"ContainerDied","Data":"d18a9a51ce0a815d93597ddb43047315c42f6b0a592908663ebd948ecf9d9048"}
Jul 14 01:45:03.989087 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.989006    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d18a9a51ce0a815d93597ddb43047315c42f6b0a592908663ebd948ecf9d9048"
Jul 14 01:45:03.990129 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.990108    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"
Jul 14 01:45:03.990272 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.990130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5" event={"ID":"9c1bcf81-bc9b-4547-b1e4-4954f4378a09","Type":"ContainerDied","Data":"69b5c8f95e36a948e51fab07295c60fdeac5669bf1b95c3fa135a8da3459d7dd"}
Jul 14 01:45:03.990272 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.990173    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69b5c8f95e36a948e51fab07295c60fdeac5669bf1b95c3fa135a8da3459d7dd"
Jul 14 01:45:03.991584 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.991563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk" event={"ID":"a7930731-9e5f-4c93-8c71-3b867883eec5","Type":"ContainerDied","Data":"1a22c90d46d135dbb4e821279b4734abcd6bfd572d0623689bfa6448ec3ca295"}
Jul 14 01:45:03.991671 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.991612    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a22c90d46d135dbb4e821279b4734abcd6bfd572d0623689bfa6448ec3ca295"
Jul 14 01:45:03.991671 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:45:03.991578    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"
Jul 14 01:49:45.689416 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.689376    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7419-140719335659824"]
Jul 14 01:49:45.690013 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.689959    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c1bcf81-bc9b-4547-b1e4-4954f4378a09" containerName="cleanup"
Jul 14 01:49:45.690013 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.689978    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c1bcf81-bc9b-4547-b1e4-4954f4378a09" containerName="cleanup"
Jul 14 01:49:45.690013 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.689999    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7930731-9e5f-4c93-8c71-3b867883eec5" containerName="cleanup"
Jul 14 01:49:45.690013 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.690006    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7930731-9e5f-4c93-8c71-3b867883eec5" containerName="cleanup"
Jul 14 01:49:45.690229 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.690017    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e586fd-1f3f-4161-9d5d-103916ebd752" containerName="cleanup"
Jul 14 01:49:45.690229 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.690025    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e586fd-1f3f-4161-9d5d-103916ebd752" containerName="cleanup"
Jul 14 01:49:45.690229 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.690114    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e586fd-1f3f-4161-9d5d-103916ebd752" containerName="cleanup"
Jul 14 01:49:45.690229 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.690124    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c1bcf81-bc9b-4547-b1e4-4954f4378a09" containerName="cleanup"
Jul 14 01:49:45.690229 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.690134    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7930731-9e5f-4c93-8c71-3b867883eec5" containerName="cleanup"
Jul 14 01:49:45.693268 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.693247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:45.695699 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.695679    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-xpbqm\""
Jul 14 01:49:45.698060 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.698036    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719335659824"]
Jul 14 01:49:45.742334 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.742306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f4d9\" (UniqueName: \"kubernetes.io/projected/f33b7a69-4f06-4196-a43f-731168ee10ba-kube-api-access-5f4d9\") pod \"test-curl-7419-140719335659824\" (UID: \"f33b7a69-4f06-4196-a43f-731168ee10ba\") " pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:45.843672 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.843646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f4d9\" (UniqueName: \"kubernetes.io/projected/f33b7a69-4f06-4196-a43f-731168ee10ba-kube-api-access-5f4d9\") pod \"test-curl-7419-140719335659824\" (UID: \"f33b7a69-4f06-4196-a43f-731168ee10ba\") " pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:45.851657 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:45.851635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f4d9\" (UniqueName: \"kubernetes.io/projected/f33b7a69-4f06-4196-a43f-731168ee10ba-kube-api-access-5f4d9\") pod \"test-curl-7419-140719335659824\" (UID: \"f33b7a69-4f06-4196-a43f-731168ee10ba\") " pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:46.003289 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:46.003263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:46.136720 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:46.135813    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719335659824"]
Jul 14 01:49:46.139219 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:49:46.139181    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf33b7a69_4f06_4196_a43f_731168ee10ba.slice/crio-32c7ac755efce7c6831d3e3098c228675efe339f18382552cd4d184701f70970 WatchSource:0}: Error finding container 32c7ac755efce7c6831d3e3098c228675efe339f18382552cd4d184701f70970: Status 404 returned error can't find the container with id 32c7ac755efce7c6831d3e3098c228675efe339f18382552cd4d184701f70970
Jul 14 01:49:46.140538 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:46.140507    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 01:49:46.410597 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:46.410509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719335659824" event={"ID":"f33b7a69-4f06-4196-a43f-731168ee10ba","Type":"ContainerStarted","Data":"32c7ac755efce7c6831d3e3098c228675efe339f18382552cd4d184701f70970"}
Jul 14 01:49:47.419129 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:47.419092    2580 generic.go:356] "Generic (PLEG): container finished" podID="f33b7a69-4f06-4196-a43f-731168ee10ba" containerID="45fafbd86bdbf01aaf0ce48fc40f1f56c648db88a72726fe62ee0f6319143d7a" exitCode=0
Jul 14 01:49:47.419538 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:47.419145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719335659824" event={"ID":"f33b7a69-4f06-4196-a43f-731168ee10ba","Type":"ContainerDied","Data":"45fafbd86bdbf01aaf0ce48fc40f1f56c648db88a72726fe62ee0f6319143d7a"}
Jul 14 01:49:48.563129 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.563106    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:48.593815 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.593788    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7419-140719335659824_f33b7a69-4f06-4196-a43f-731168ee10ba/test-curl-7419-140719335659824/0.log"
Jul 14 01:49:48.616530 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.616503    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7419-140719335659824"]
Jul 14 01:49:48.620010 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.619983    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7419-140719335659824"]
Jul 14 01:49:48.668684 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.668660    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5f4d9\" (UniqueName: \"kubernetes.io/projected/f33b7a69-4f06-4196-a43f-731168ee10ba-kube-api-access-5f4d9\") pod \"f33b7a69-4f06-4196-a43f-731168ee10ba\" (UID: \"f33b7a69-4f06-4196-a43f-731168ee10ba\") "
Jul 14 01:49:48.670888 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.670862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f33b7a69-4f06-4196-a43f-731168ee10ba-kube-api-access-5f4d9" (OuterVolumeSpecName: "kube-api-access-5f4d9") pod "f33b7a69-4f06-4196-a43f-731168ee10ba" (UID: "f33b7a69-4f06-4196-a43f-731168ee10ba"). InnerVolumeSpecName "kube-api-access-5f4d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:49:48.744140 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.744114    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7419-140719333733424"]
Jul 14 01:49:48.744500 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.744488    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33b7a69-4f06-4196-a43f-731168ee10ba" containerName="test-curl-7419-140719335659824"
Jul 14 01:49:48.744557 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.744501    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33b7a69-4f06-4196-a43f-731168ee10ba" containerName="test-curl-7419-140719335659824"
Jul 14 01:49:48.744594 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.744575    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f33b7a69-4f06-4196-a43f-731168ee10ba" containerName="test-curl-7419-140719335659824"
Jul 14 01:49:48.747432 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.747417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:48.751967 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.751941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333733424"]
Jul 14 01:49:48.769507 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.769482    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f4d9\" (UniqueName: \"kubernetes.io/projected/f33b7a69-4f06-4196-a43f-731168ee10ba-kube-api-access-5f4d9\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:49:48.870560 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.870509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bphvw\" (UniqueName: \"kubernetes.io/projected/80cfef76-7f04-44ab-bfca-d109290b745d-kube-api-access-bphvw\") pod \"test-curl-7419-140719333733424\" (UID: \"80cfef76-7f04-44ab-bfca-d109290b745d\") " pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:48.971765 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.971676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bphvw\" (UniqueName: \"kubernetes.io/projected/80cfef76-7f04-44ab-bfca-d109290b745d-kube-api-access-bphvw\") pod \"test-curl-7419-140719333733424\" (UID: \"80cfef76-7f04-44ab-bfca-d109290b745d\") " pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:48.979352 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:48.979324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bphvw\" (UniqueName: \"kubernetes.io/projected/80cfef76-7f04-44ab-bfca-d109290b745d-kube-api-access-bphvw\") pod \"test-curl-7419-140719333733424\" (UID: \"80cfef76-7f04-44ab-bfca-d109290b745d\") " pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:49.057139 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:49.057108    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:49.188781 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:49.188754    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333733424"]
Jul 14 01:49:49.190837 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:49:49.190813    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80cfef76_7f04_44ab_bfca_d109290b745d.slice/crio-487231c510ee67c4147203ad585b97003ef2ed8770e4627ae1b857462e68ea98 WatchSource:0}: Error finding container 487231c510ee67c4147203ad585b97003ef2ed8770e4627ae1b857462e68ea98: Status 404 returned error can't find the container with id 487231c510ee67c4147203ad585b97003ef2ed8770e4627ae1b857462e68ea98
Jul 14 01:49:49.431558 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:49.431503    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333733424" event={"ID":"80cfef76-7f04-44ab-bfca-d109290b745d","Type":"ContainerStarted","Data":"487231c510ee67c4147203ad585b97003ef2ed8770e4627ae1b857462e68ea98"}
Jul 14 01:49:49.432765 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:49.432743    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:49.432863 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:49.432779    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32c7ac755efce7c6831d3e3098c228675efe339f18382552cd4d184701f70970"
Jul 14 01:49:49.442490 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:49:49.442464    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719335659824\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="f33b7a69-4f06-4196-a43f-731168ee10ba" pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:49.640854 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:49:49.640824    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719335659824\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="f33b7a69-4f06-4196-a43f-731168ee10ba" pod="openshift-ingress/test-curl-7419-140719335659824"
Jul 14 01:49:49.642787 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:49.642767    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f33b7a69-4f06-4196-a43f-731168ee10ba" path="/var/lib/kubelet/pods/f33b7a69-4f06-4196-a43f-731168ee10ba/volumes"
Jul 14 01:49:50.439585 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:50.439551    2580 generic.go:356] "Generic (PLEG): container finished" podID="80cfef76-7f04-44ab-bfca-d109290b745d" containerID="bb20b6a65033ed53244c14669ab509ea26f8fe2cefe184ca711154a8a4e313a8" exitCode=0
Jul 14 01:49:50.439749 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:50.439638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333733424" event={"ID":"80cfef76-7f04-44ab-bfca-d109290b745d","Type":"ContainerDied","Data":"bb20b6a65033ed53244c14669ab509ea26f8fe2cefe184ca711154a8a4e313a8"}
Jul 14 01:49:51.585742 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.585721    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:51.608310 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.608291    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7419-140719333733424_80cfef76-7f04-44ab-bfca-d109290b745d/test-curl-7419-140719333733424/0.log"
Jul 14 01:49:51.643936 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.643912    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7419-140719333733424"]
Jul 14 01:49:51.644032 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.643947    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7419-140719333733424"]
Jul 14 01:49:51.695012 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.694984    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bphvw\" (UniqueName: \"kubernetes.io/projected/80cfef76-7f04-44ab-bfca-d109290b745d-kube-api-access-bphvw\") pod \"80cfef76-7f04-44ab-bfca-d109290b745d\" (UID: \"80cfef76-7f04-44ab-bfca-d109290b745d\") "
Jul 14 01:49:51.697104 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.697055    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80cfef76-7f04-44ab-bfca-d109290b745d-kube-api-access-bphvw" (OuterVolumeSpecName: "kube-api-access-bphvw") pod "80cfef76-7f04-44ab-bfca-d109290b745d" (UID: "80cfef76-7f04-44ab-bfca-d109290b745d"). InnerVolumeSpecName "kube-api-access-bphvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:49:51.796121 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.796091    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bphvw\" (UniqueName: \"kubernetes.io/projected/80cfef76-7f04-44ab-bfca-d109290b745d-kube-api-access-bphvw\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:49:51.800563 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.800539    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:51.800910 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.800899    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80cfef76-7f04-44ab-bfca-d109290b745d" containerName="test-curl-7419-140719333733424"
Jul 14 01:49:51.800954 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.800912    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="80cfef76-7f04-44ab-bfca-d109290b745d" containerName="test-curl-7419-140719333733424"
Jul 14 01:49:51.800993 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.800984    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="80cfef76-7f04-44ab-bfca-d109290b745d" containerName="test-curl-7419-140719333733424"
Jul 14 01:49:51.804287 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.804273    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:51.810550 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.810516    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:51.897154 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.897129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvwhb\" (UniqueName: \"kubernetes.io/projected/049667f7-a74b-43db-96ff-fecdaa5735af-kube-api-access-pvwhb\") pod \"test-curl-7419-140719333580848\" (UID: \"049667f7-a74b-43db-96ff-fecdaa5735af\") " pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:51.998176 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:51.998154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvwhb\" (UniqueName: \"kubernetes.io/projected/049667f7-a74b-43db-96ff-fecdaa5735af-kube-api-access-pvwhb\") pod \"test-curl-7419-140719333580848\" (UID: \"049667f7-a74b-43db-96ff-fecdaa5735af\") " pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:52.007687 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:52.007662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvwhb\" (UniqueName: \"kubernetes.io/projected/049667f7-a74b-43db-96ff-fecdaa5735af-kube-api-access-pvwhb\") pod \"test-curl-7419-140719333580848\" (UID: \"049667f7-a74b-43db-96ff-fecdaa5735af\") " pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:52.113874 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:52.113846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:52.249793 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:52.249765    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:52.250478 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:49:52.250450    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod049667f7_a74b_43db_96ff_fecdaa5735af.slice/crio-e3f79cce42ca6158be7e8a634b4cf2e3fe8f5c174c260649dbd60c322d17d405 WatchSource:0}: Error finding container e3f79cce42ca6158be7e8a634b4cf2e3fe8f5c174c260649dbd60c322d17d405: Status 404 returned error can't find the container with id e3f79cce42ca6158be7e8a634b4cf2e3fe8f5c174c260649dbd60c322d17d405
Jul 14 01:49:52.452526 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:52.452497    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:52.452660 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:52.452549    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="487231c510ee67c4147203ad585b97003ef2ed8770e4627ae1b857462e68ea98"
Jul 14 01:49:52.453777 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:52.453748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333580848" event={"ID":"049667f7-a74b-43db-96ff-fecdaa5735af","Type":"ContainerStarted","Data":"e3f79cce42ca6158be7e8a634b4cf2e3fe8f5c174c260649dbd60c322d17d405"}
Jul 14 01:49:52.462442 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:49:52.462416    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719333733424\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="80cfef76-7f04-44ab-bfca-d109290b745d" pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:53.460080 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:53.460045    2580 generic.go:356] "Generic (PLEG): container finished" podID="049667f7-a74b-43db-96ff-fecdaa5735af" containerID="d875e9e8fb7d365192dd3fc737287207226d0d12bd69f4df6f9225bc088ba9a5" exitCode=0
Jul 14 01:49:53.460430 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:53.460101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333580848" event={"ID":"049667f7-a74b-43db-96ff-fecdaa5735af","Type":"ContainerDied","Data":"d875e9e8fb7d365192dd3fc737287207226d0d12bd69f4df6f9225bc088ba9a5"}
Jul 14 01:49:53.475009 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:49:53.474976    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719333733424\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="80cfef76-7f04-44ab-bfca-d109290b745d" pod="openshift-ingress/test-curl-7419-140719333733424"
Jul 14 01:49:53.642093 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:53.642061    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80cfef76-7f04-44ab-bfca-d109290b745d" path="/var/lib/kubelet/pods/80cfef76-7f04-44ab-bfca-d109290b745d/volumes"
Jul 14 01:49:54.602822 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.602798    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:54.622881 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.622855    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7419-140719333580848_049667f7-a74b-43db-96ff-fecdaa5735af/test-curl-7419-140719333580848/0.log"
Jul 14 01:49:54.646639 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.646609    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:54.648977 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.648955    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:54.719318 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.719255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvwhb\" (UniqueName: \"kubernetes.io/projected/049667f7-a74b-43db-96ff-fecdaa5735af-kube-api-access-pvwhb\") pod \"049667f7-a74b-43db-96ff-fecdaa5735af\" (UID: \"049667f7-a74b-43db-96ff-fecdaa5735af\") "
Jul 14 01:49:54.721346 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.721322    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049667f7-a74b-43db-96ff-fecdaa5735af-kube-api-access-pvwhb" (OuterVolumeSpecName: "kube-api-access-pvwhb") pod "049667f7-a74b-43db-96ff-fecdaa5735af" (UID: "049667f7-a74b-43db-96ff-fecdaa5735af"). InnerVolumeSpecName "kube-api-access-pvwhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:49:54.768209 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.768183    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:54.768591 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.768578    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="049667f7-a74b-43db-96ff-fecdaa5735af" containerName="test-curl-7419-140719333580848"
Jul 14 01:49:54.768642 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.768594    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="049667f7-a74b-43db-96ff-fecdaa5735af" containerName="test-curl-7419-140719333580848"
Jul 14 01:49:54.768693 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.768683    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="049667f7-a74b-43db-96ff-fecdaa5735af" containerName="test-curl-7419-140719333580848"
Jul 14 01:49:54.771554 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.771540    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:54.786452 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.786430    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:54.820200 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.820174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mmn4\" (UniqueName: \"kubernetes.io/projected/db9cac3a-cdec-4cd1-acc6-77db46352376-kube-api-access-4mmn4\") pod \"test-curl-7419-140719333580848\" (UID: \"db9cac3a-cdec-4cd1-acc6-77db46352376\") " pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:54.820292 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.820233    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvwhb\" (UniqueName: \"kubernetes.io/projected/049667f7-a74b-43db-96ff-fecdaa5735af-kube-api-access-pvwhb\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:49:54.920664 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.920641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mmn4\" (UniqueName: \"kubernetes.io/projected/db9cac3a-cdec-4cd1-acc6-77db46352376-kube-api-access-4mmn4\") pod \"test-curl-7419-140719333580848\" (UID: \"db9cac3a-cdec-4cd1-acc6-77db46352376\") " pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:54.928199 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:54.928176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mmn4\" (UniqueName: \"kubernetes.io/projected/db9cac3a-cdec-4cd1-acc6-77db46352376-kube-api-access-4mmn4\") pod \"test-curl-7419-140719333580848\" (UID: \"db9cac3a-cdec-4cd1-acc6-77db46352376\") " pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:55.079917 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:55.079886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:55.212093 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:55.212074    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:55.214380 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:49:55.214348    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb9cac3a_cdec_4cd1_acc6_77db46352376.slice/crio-a947ccf2b96b6ab6c26f7af291baade06b708337bb28eb3f22d5463aa9c7c149 WatchSource:0}: Error finding container a947ccf2b96b6ab6c26f7af291baade06b708337bb28eb3f22d5463aa9c7c149: Status 404 returned error can't find the container with id a947ccf2b96b6ab6c26f7af291baade06b708337bb28eb3f22d5463aa9c7c149
Jul 14 01:49:55.471407 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:55.471335    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:55.471570 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:55.471333    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3f79cce42ca6158be7e8a634b4cf2e3fe8f5c174c260649dbd60c322d17d405"
Jul 14 01:49:55.472644 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:55.472616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333580848" event={"ID":"db9cac3a-cdec-4cd1-acc6-77db46352376","Type":"ContainerStarted","Data":"a947ccf2b96b6ab6c26f7af291baade06b708337bb28eb3f22d5463aa9c7c149"}
Jul 14 01:49:55.482732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:55.482705    2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7419-140719333580848" oldPodUID="049667f7-a74b-43db-96ff-fecdaa5735af" podUID="db9cac3a-cdec-4cd1-acc6-77db46352376"
Jul 14 01:49:55.642183 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:55.642158    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="049667f7-a74b-43db-96ff-fecdaa5735af" path="/var/lib/kubelet/pods/049667f7-a74b-43db-96ff-fecdaa5735af/volumes"
Jul 14 01:49:56.480092 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:56.480055    2580 generic.go:356] "Generic (PLEG): container finished" podID="db9cac3a-cdec-4cd1-acc6-77db46352376" containerID="4991caa0dd7ad727cabc9bc2bd060d682137c74f75cdde4a56a7591f6eb3f11a" exitCode=0
Jul 14 01:49:56.480258 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:56.480143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333580848" event={"ID":"db9cac3a-cdec-4cd1-acc6-77db46352376","Type":"ContainerDied","Data":"4991caa0dd7ad727cabc9bc2bd060d682137c74f75cdde4a56a7591f6eb3f11a"}
Jul 14 01:49:57.628968 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.628945    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:57.646202 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.646162    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7419-140719333580848_db9cac3a-cdec-4cd1-acc6-77db46352376/test-curl-7419-140719333580848/0.log"
Jul 14 01:49:57.669354 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.669325    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:57.672539 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.672503    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7419-140719333580848"]
Jul 14 01:49:57.744425 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.744402    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mmn4\" (UniqueName: \"kubernetes.io/projected/db9cac3a-cdec-4cd1-acc6-77db46352376-kube-api-access-4mmn4\") pod \"db9cac3a-cdec-4cd1-acc6-77db46352376\" (UID: \"db9cac3a-cdec-4cd1-acc6-77db46352376\") "
Jul 14 01:49:57.746589 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.746561    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db9cac3a-cdec-4cd1-acc6-77db46352376-kube-api-access-4mmn4" (OuterVolumeSpecName: "kube-api-access-4mmn4") pod "db9cac3a-cdec-4cd1-acc6-77db46352376" (UID: "db9cac3a-cdec-4cd1-acc6-77db46352376"). InnerVolumeSpecName "kube-api-access-4mmn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:49:57.787657 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.787633    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:49:57.788009 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.787996    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9cac3a-cdec-4cd1-acc6-77db46352376" containerName="test-curl-7419-140719333580848"
Jul 14 01:49:57.788052 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.788010    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9cac3a-cdec-4cd1-acc6-77db46352376" containerName="test-curl-7419-140719333580848"
Jul 14 01:49:57.788087 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.788076    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db9cac3a-cdec-4cd1-acc6-77db46352376" containerName="test-curl-7419-140719333580848"
Jul 14 01:49:57.790951 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.790928    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:49:57.793867 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.793846    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:49:57.845012 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.844984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfr4r\" (UniqueName: \"kubernetes.io/projected/ef563c31-a50d-4d84-bca7-abe2b14aa581-kube-api-access-vfr4r\") pod \"test-curl-7419-140719333581856\" (UID: \"ef563c31-a50d-4d84-bca7-abe2b14aa581\") " pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:49:57.845117 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.845060    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mmn4\" (UniqueName: \"kubernetes.io/projected/db9cac3a-cdec-4cd1-acc6-77db46352376-kube-api-access-4mmn4\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:49:57.946246 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.946220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfr4r\" (UniqueName: \"kubernetes.io/projected/ef563c31-a50d-4d84-bca7-abe2b14aa581-kube-api-access-vfr4r\") pod \"test-curl-7419-140719333581856\" (UID: \"ef563c31-a50d-4d84-bca7-abe2b14aa581\") " pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:49:57.953470 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:57.953451    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfr4r\" (UniqueName: \"kubernetes.io/projected/ef563c31-a50d-4d84-bca7-abe2b14aa581-kube-api-access-vfr4r\") pod \"test-curl-7419-140719333581856\" (UID: \"ef563c31-a50d-4d84-bca7-abe2b14aa581\") " pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:49:58.100697 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:58.100635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:49:58.231392 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:58.231369    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:49:58.233673 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:49:58.233645    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef563c31_a50d_4d84_bca7_abe2b14aa581.slice/crio-ab19af140664ea47f76336122cd0e91062ead5dee548f12b5932de872d829a58 WatchSource:0}: Error finding container ab19af140664ea47f76336122cd0e91062ead5dee548f12b5932de872d829a58: Status 404 returned error can't find the container with id ab19af140664ea47f76336122cd0e91062ead5dee548f12b5932de872d829a58
Jul 14 01:49:58.491921 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:58.491856    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:58.491921 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:58.491890    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a947ccf2b96b6ab6c26f7af291baade06b708337bb28eb3f22d5463aa9c7c149"
Jul 14 01:49:58.493252 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:58.493229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333581856" event={"ID":"ef563c31-a50d-4d84-bca7-abe2b14aa581","Type":"ContainerStarted","Data":"ab19af140664ea47f76336122cd0e91062ead5dee548f12b5932de872d829a58"}
Jul 14 01:49:58.502894 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:49:58.502869    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719333580848\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="db9cac3a-cdec-4cd1-acc6-77db46352376" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:59.499709 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:59.499677    2580 generic.go:356] "Generic (PLEG): container finished" podID="ef563c31-a50d-4d84-bca7-abe2b14aa581" containerID="fe3bcd65a1620460b11ebc76e340e272ffef7994616d9965aefdfdb2e6774860" exitCode=0
Jul 14 01:49:59.500105 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:59.499762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333581856" event={"ID":"ef563c31-a50d-4d84-bca7-abe2b14aa581","Type":"ContainerDied","Data":"fe3bcd65a1620460b11ebc76e340e272ffef7994616d9965aefdfdb2e6774860"}
Jul 14 01:49:59.513672 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:49:59.513637    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719333580848\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="db9cac3a-cdec-4cd1-acc6-77db46352376" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:59.641756 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:49:59.641725    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719333580848\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="db9cac3a-cdec-4cd1-acc6-77db46352376" pod="openshift-ingress/test-curl-7419-140719333580848"
Jul 14 01:49:59.642734 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:49:59.642714    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db9cac3a-cdec-4cd1-acc6-77db46352376" path="/var/lib/kubelet/pods/db9cac3a-cdec-4cd1-acc6-77db46352376/volumes"
Jul 14 01:50:00.645167 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.645145    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:00.667355 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.667328    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7419-140719333581856_ef563c31-a50d-4d84-bca7-abe2b14aa581/test-curl-7419-140719333581856/0.log"
Jul 14 01:50:00.695289 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.695268    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:50:00.699463 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.699438    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:50:00.771468 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.771407    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfr4r\" (UniqueName: \"kubernetes.io/projected/ef563c31-a50d-4d84-bca7-abe2b14aa581-kube-api-access-vfr4r\") pod \"ef563c31-a50d-4d84-bca7-abe2b14aa581\" (UID: \"ef563c31-a50d-4d84-bca7-abe2b14aa581\") "
Jul 14 01:50:00.773512 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.773488    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef563c31-a50d-4d84-bca7-abe2b14aa581-kube-api-access-vfr4r" (OuterVolumeSpecName: "kube-api-access-vfr4r") pod "ef563c31-a50d-4d84-bca7-abe2b14aa581" (UID: "ef563c31-a50d-4d84-bca7-abe2b14aa581"). InnerVolumeSpecName "kube-api-access-vfr4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:50:00.820378 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.820352    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:50:00.820759 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.820746    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef563c31-a50d-4d84-bca7-abe2b14aa581" containerName="test-curl-7419-140719333581856"
Jul 14 01:50:00.820808 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.820761    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef563c31-a50d-4d84-bca7-abe2b14aa581" containerName="test-curl-7419-140719333581856"
Jul 14 01:50:00.820848 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.820841    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef563c31-a50d-4d84-bca7-abe2b14aa581" containerName="test-curl-7419-140719333581856"
Jul 14 01:50:00.823792 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.823778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:00.830045 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.830021    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:50:00.872410 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.872382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgpch\" (UniqueName: \"kubernetes.io/projected/3414b654-a808-42ef-8860-7f7c87c5b282-kube-api-access-wgpch\") pod \"test-curl-7419-140719333581856\" (UID: \"3414b654-a808-42ef-8860-7f7c87c5b282\") " pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:00.872541 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.872426    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfr4r\" (UniqueName: \"kubernetes.io/projected/ef563c31-a50d-4d84-bca7-abe2b14aa581-kube-api-access-vfr4r\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:50:00.973185 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.973158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgpch\" (UniqueName: \"kubernetes.io/projected/3414b654-a808-42ef-8860-7f7c87c5b282-kube-api-access-wgpch\") pod \"test-curl-7419-140719333581856\" (UID: \"3414b654-a808-42ef-8860-7f7c87c5b282\") " pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:00.981482 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:00.981464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgpch\" (UniqueName: \"kubernetes.io/projected/3414b654-a808-42ef-8860-7f7c87c5b282-kube-api-access-wgpch\") pod \"test-curl-7419-140719333581856\" (UID: \"3414b654-a808-42ef-8860-7f7c87c5b282\") " pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:01.133239 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:01.133185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:01.284446 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:01.284417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:50:01.285986 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:50:01.285962    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3414b654_a808_42ef_8860_7f7c87c5b282.slice/crio-6716baf46d6610b1ddd6d74589e17e0db31e316415332b620c0a3ac8262d2c6b WatchSource:0}: Error finding container 6716baf46d6610b1ddd6d74589e17e0db31e316415332b620c0a3ac8262d2c6b: Status 404 returned error can't find the container with id 6716baf46d6610b1ddd6d74589e17e0db31e316415332b620c0a3ac8262d2c6b
Jul 14 01:50:01.511545 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:01.511511    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:01.511699 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:01.511556    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab19af140664ea47f76336122cd0e91062ead5dee548f12b5932de872d829a58"
Jul 14 01:50:01.512770 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:01.512747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333581856" event={"ID":"3414b654-a808-42ef-8860-7f7c87c5b282","Type":"ContainerStarted","Data":"6716baf46d6610b1ddd6d74589e17e0db31e316415332b620c0a3ac8262d2c6b"}
Jul 14 01:50:01.522613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:01.522592    2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7419-140719333581856" oldPodUID="ef563c31-a50d-4d84-bca7-abe2b14aa581" podUID="3414b654-a808-42ef-8860-7f7c87c5b282"
Jul 14 01:50:01.644209 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:01.644185    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef563c31-a50d-4d84-bca7-abe2b14aa581" path="/var/lib/kubelet/pods/ef563c31-a50d-4d84-bca7-abe2b14aa581/volumes"
Jul 14 01:50:02.519813 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:02.519780    2580 generic.go:356] "Generic (PLEG): container finished" podID="3414b654-a808-42ef-8860-7f7c87c5b282" containerID="68efd6b8574c798da6650dcb3490040fc089083f45712064c7f82b9729a44f92" exitCode=0
Jul 14 01:50:02.520192 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:02.519866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7419-140719333581856" event={"ID":"3414b654-a808-42ef-8860-7f7c87c5b282","Type":"ContainerDied","Data":"68efd6b8574c798da6650dcb3490040fc089083f45712064c7f82b9729a44f92"}
Jul 14 01:50:03.671304 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.671282    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:03.689815 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.689793    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7419-140719333581856_3414b654-a808-42ef-8860-7f7c87c5b282/test-curl-7419-140719333581856/0.log"
Jul 14 01:50:03.712740 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.712716    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:50:03.718128 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.718108    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7419-140719333581856"]
Jul 14 01:50:03.798941 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.798885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgpch\" (UniqueName: \"kubernetes.io/projected/3414b654-a808-42ef-8860-7f7c87c5b282-kube-api-access-wgpch\") pod \"3414b654-a808-42ef-8860-7f7c87c5b282\" (UID: \"3414b654-a808-42ef-8860-7f7c87c5b282\") "
Jul 14 01:50:03.800994 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.800967    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3414b654-a808-42ef-8860-7f7c87c5b282-kube-api-access-wgpch" (OuterVolumeSpecName: "kube-api-access-wgpch") pod "3414b654-a808-42ef-8860-7f7c87c5b282" (UID: "3414b654-a808-42ef-8860-7f7c87c5b282"). InnerVolumeSpecName "kube-api-access-wgpch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 01:50:03.896262 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.896227    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"]
Jul 14 01:50:03.896858 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.896841    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3414b654-a808-42ef-8860-7f7c87c5b282" containerName="test-curl-7419-140719333581856"
Jul 14 01:50:03.896918 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.896863    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3414b654-a808-42ef-8860-7f7c87c5b282" containerName="test-curl-7419-140719333581856"
Jul 14 01:50:03.896992 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.896980    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3414b654-a808-42ef-8860-7f7c87c5b282" containerName="test-curl-7419-140719333581856"
Jul 14 01:50:03.899798 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.899781    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgpch\" (UniqueName: \"kubernetes.io/projected/3414b654-a808-42ef-8860-7f7c87c5b282-kube-api-access-wgpch\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:50:03.900512 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.900496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"
Jul 14 01:50:03.902546 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.902499    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:50:03.904933 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:03.904910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"]
Jul 14 01:50:04.000280 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:04.000251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9343b44c-932e-42bf-befe-36bd8440cfdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc\" (UID: \"9343b44c-932e-42bf-befe-36bd8440cfdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"
Jul 14 01:50:04.101669 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:04.101598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9343b44c-932e-42bf-befe-36bd8440cfdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc\" (UID: \"9343b44c-932e-42bf-befe-36bd8440cfdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"
Jul 14 01:50:04.102126 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:04.102108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9343b44c-932e-42bf-befe-36bd8440cfdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc\" (UID: \"9343b44c-932e-42bf-befe-36bd8440cfdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"
Jul 14 01:50:04.214636 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:04.214612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"
Jul 14 01:50:04.351251 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:04.351225    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"]
Jul 14 01:50:04.353378 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:50:04.353322    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9343b44c_932e_42bf_befe_36bd8440cfdb.slice/crio-54f0e5bd9292742b11ccba578ea44621320064a2bcf2764dcf199b908404b733 WatchSource:0}: Error finding container 54f0e5bd9292742b11ccba578ea44621320064a2bcf2764dcf199b908404b733: Status 404 returned error can't find the container with id 54f0e5bd9292742b11ccba578ea44621320064a2bcf2764dcf199b908404b733
Jul 14 01:50:04.531882 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:04.531838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc" event={"ID":"9343b44c-932e-42bf-befe-36bd8440cfdb","Type":"ContainerStarted","Data":"54f0e5bd9292742b11ccba578ea44621320064a2bcf2764dcf199b908404b733"}
Jul 14 01:50:04.533045 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:04.533025    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6716baf46d6610b1ddd6d74589e17e0db31e316415332b620c0a3ac8262d2c6b"
Jul 14 01:50:04.533143 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:04.533051    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:04.542033 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:50:04.542006    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719333581856\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="3414b654-a808-42ef-8860-7f7c87c5b282" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:05.540788 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:05.540753    2580 generic.go:356] "Generic (PLEG): container finished" podID="9343b44c-932e-42bf-befe-36bd8440cfdb" containerID="39f7829bca05c7e9abfe7f319639f749af362b140caf9cb076d343d6df4c5da3" exitCode=0
Jul 14 01:50:05.541218 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:05.540828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc" event={"ID":"9343b44c-932e-42bf-befe-36bd8440cfdb","Type":"ContainerDied","Data":"39f7829bca05c7e9abfe7f319639f749af362b140caf9cb076d343d6df4c5da3"}
Jul 14 01:50:05.542376 ip-10-0-141-32 kubenswrapper[2580]: E0714 01:50:05.542356    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-140719333581856\" is forbidden: User \"system:node:ip-10-0-141-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-32.ec2.internal' and this object" podUID="3414b654-a808-42ef-8860-7f7c87c5b282" pod="openshift-ingress/test-curl-7419-140719333581856"
Jul 14 01:50:05.642665 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:05.642636    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3414b654-a808-42ef-8860-7f7c87c5b282" path="/var/lib/kubelet/pods/3414b654-a808-42ef-8860-7f7c87c5b282/volumes"
Jul 14 01:50:06.678093 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:06.678070    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"
Jul 14 01:50:06.825155 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:06.825087    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9343b44c-932e-42bf-befe-36bd8440cfdb-maas-api-service-ca\") pod \"9343b44c-932e-42bf-befe-36bd8440cfdb\" (UID: \"9343b44c-932e-42bf-befe-36bd8440cfdb\") "
Jul 14 01:50:06.825421 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:06.825403    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9343b44c-932e-42bf-befe-36bd8440cfdb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9343b44c-932e-42bf-befe-36bd8440cfdb" (UID: "9343b44c-932e-42bf-befe-36bd8440cfdb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:50:06.926153 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:06.926127    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9343b44c-932e-42bf-befe-36bd8440cfdb-maas-api-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:50:07.552998 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:07.552972    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc"
Jul 14 01:50:07.553162 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:07.553013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc" event={"ID":"9343b44c-932e-42bf-befe-36bd8440cfdb","Type":"ContainerDied","Data":"54f0e5bd9292742b11ccba578ea44621320064a2bcf2764dcf199b908404b733"}
Jul 14 01:50:07.553162 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:07.553056    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54f0e5bd9292742b11ccba578ea44621320064a2bcf2764dcf199b908404b733"
Jul 14 01:50:14.464573 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:14.464539    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"]
Jul 14 01:50:14.468943 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:14.468916    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-99e8d0-29733225-mnsqk"]
Jul 14 01:50:15.641354 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:15.641314    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7930731-9e5f-4c93-8c71-3b867883eec5" path="/var/lib/kubelet/pods/a7930731-9e5f-4c93-8c71-3b867883eec5/volumes"
Jul 14 01:50:30.096552 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.096496    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"]
Jul 14 01:50:30.097106 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.097087    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9343b44c-932e-42bf-befe-36bd8440cfdb" containerName="revoke-keys"
Jul 14 01:50:30.097153 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.097110    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9343b44c-932e-42bf-befe-36bd8440cfdb" containerName="revoke-keys"
Jul 14 01:50:30.097211 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.097200    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9343b44c-932e-42bf-befe-36bd8440cfdb" containerName="revoke-keys"
Jul 14 01:50:30.100861 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.100840    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"
Jul 14 01:50:30.103129 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.103108    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kh8h4\""
Jul 14 01:50:30.105394 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.105365    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"]
Jul 14 01:50:30.216895 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.216866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3b8e15-313d-4152-ac86-b367dc26bdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw\" (UID: \"4a3b8e15-313d-4152-ac86-b367dc26bdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"
Jul 14 01:50:30.317868 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.317837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3b8e15-313d-4152-ac86-b367dc26bdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw\" (UID: \"4a3b8e15-313d-4152-ac86-b367dc26bdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"
Jul 14 01:50:30.318405 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.318387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3b8e15-313d-4152-ac86-b367dc26bdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw\" (UID: \"4a3b8e15-313d-4152-ac86-b367dc26bdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"
Jul 14 01:50:30.413181 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.413113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"
Jul 14 01:50:30.548058 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.548035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"]
Jul 14 01:50:30.550196 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:50:30.550165    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3b8e15_313d_4152_ac86_b367dc26bdb7.slice/crio-98f96fe363646bea2d5a0d280fb42bea65cf1721031adb7f10c6b7f6478fe827 WatchSource:0}: Error finding container 98f96fe363646bea2d5a0d280fb42bea65cf1721031adb7f10c6b7f6478fe827: Status 404 returned error can't find the container with id 98f96fe363646bea2d5a0d280fb42bea65cf1721031adb7f10c6b7f6478fe827
Jul 14 01:50:30.673406 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:30.673326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw" event={"ID":"4a3b8e15-313d-4152-ac86-b367dc26bdb7","Type":"ContainerStarted","Data":"98f96fe363646bea2d5a0d280fb42bea65cf1721031adb7f10c6b7f6478fe827"}
Jul 14 01:50:31.680996 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:31.680911    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a3b8e15-313d-4152-ac86-b367dc26bdb7" containerID="aca8c090e10bbd7150e3f97c36589376e096103bab4aa6d1a9800858bdffc3bf" exitCode=0
Jul 14 01:50:31.681340 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:31.680993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw" event={"ID":"4a3b8e15-313d-4152-ac86-b367dc26bdb7","Type":"ContainerDied","Data":"aca8c090e10bbd7150e3f97c36589376e096103bab4aa6d1a9800858bdffc3bf"}
Jul 14 01:50:32.819534 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:32.819503    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"
Jul 14 01:50:32.942619 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:32.942555    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3b8e15-313d-4152-ac86-b367dc26bdb7-maas-api-service-ca\") pod \"4a3b8e15-313d-4152-ac86-b367dc26bdb7\" (UID: \"4a3b8e15-313d-4152-ac86-b367dc26bdb7\") "
Jul 14 01:50:32.942867 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:32.942847    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a3b8e15-313d-4152-ac86-b367dc26bdb7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a3b8e15-313d-4152-ac86-b367dc26bdb7" (UID: "4a3b8e15-313d-4152-ac86-b367dc26bdb7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 01:50:33.043767 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:33.043744    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3b8e15-313d-4152-ac86-b367dc26bdb7-maas-api-service-ca\") on node \"ip-10-0-141-32.ec2.internal\" DevicePath \"\""
Jul 14 01:50:33.691663 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:33.691634    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw"
Jul 14 01:50:33.691806 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:33.691679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw" event={"ID":"4a3b8e15-313d-4152-ac86-b367dc26bdb7","Type":"ContainerDied","Data":"98f96fe363646bea2d5a0d280fb42bea65cf1721031adb7f10c6b7f6478fe827"}
Jul 14 01:50:33.691806 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:33.691730    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98f96fe363646bea2d5a0d280fb42bea65cf1721031adb7f10c6b7f6478fe827"
Jul 14 01:50:40.674033 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:40.673988    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"]
Jul 14 01:50:40.676876 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:40.676854    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e42ce1-29733225-86jb5"]
Jul 14 01:50:41.642054 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:50:41.642019    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c1bcf81-bc9b-4547-b1e4-4954f4378a09" path="/var/lib/kubelet/pods/9c1bcf81-bc9b-4547-b1e4-4954f4378a09/volumes"
Jul 14 01:51:01.674849 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:01.674748    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75868574bf-zknhx_6dd08212-b3b0-4e6e-b575-b745166eb21e/manager/0.log"
Jul 14 01:51:02.135954 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:02.135927    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc876bb4c-jc6xk_a19ea936-a202-4f6b-8d8d-95d5c1a2bab7/copy-manifests/0.log"
Jul 14 01:51:02.155899 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:02.155860    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc876bb4c-jc6xk_a19ea936-a202-4f6b-8d8d-95d5c1a2bab7/manager/0.log"
Jul 14 01:51:02.629735 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:02.629706    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-74f8fc7886-92hx5_84e427c6-0e76-41c2-8eed-c5a98f65d954/maas-api/0.log"
Jul 14 01:51:02.843817 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:02.843792    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733210-2zhvm_d5b0626d-8472-4371-881d-5d04faf65463/cleanup/0.log"
Jul 14 01:51:02.950792 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:02.950721    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733225-wmfv5_75e586fd-1f3f-4161-9d5d-103916ebd752/cleanup/0.log"
Jul 14 01:51:03.271089 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:03.271060    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-597948-m4rvh_90b50e27-cc85-4237-9905-dcf5aaa8aae5/revoke-keys/0.log"
Jul 14 01:51:03.374200 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:03.374178    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-7b8e5a-bqvj2_50aed537-dbdc-494a-8171-59abcd449038/revoke-keys/0.log"
Jul 14 01:51:03.595593 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:03.595511    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-8ddee0-8fkv6_5887c777-be21-474e-a52b-1795e2785ef5/revoke-keys/0.log"
Jul 14 01:51:03.709055 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:03.709023    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-0c7042-h5bgm_6e2b2514-5f43-42fd-85b8-cedca21f2922/revoke-keys/0.log"
Jul 14 01:51:03.935136 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:03.935063    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-b02b4cdc-4mtxt_1bf44989-b22f-4b03-90c2-6ac6edb88dfc/revoke-keys/0.log"
Jul 14 01:51:04.153190 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:04.153159    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-99e8d0-nvbnc_9343b44c-932e-42bf-befe-36bd8440cfdb/revoke-keys/0.log"
Jul 14 01:51:04.262880 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:04.262855    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-e42ce1-blpgw_4a3b8e15-313d-4152-ac86-b367dc26bdb7/revoke-keys/0.log"
Jul 14 01:51:04.370885 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:04.370864    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-7kmrm_cc82c55a-c368-49a3-96a6-a08d379e2a98/postgres/0.log"
Jul 14 01:51:05.963383 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:05.963355    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-frt9g_fa784b3b-bd5f-4714-8877-816544fdfdaf/kuadrant-console-plugin/0.log"
Jul 14 01:51:07.110939 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:07.110910    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d697d898-xdr7l_72ca077f-314f-4b4d-80ba-d9f7f89fd270/kube-auth-proxy/0.log"
Jul 14 01:51:07.465689 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:07.465601    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ggjh2_4583f204-ce27-431d-8e2a-2fc74b67fd08/payload-processing/0.log"
Jul 14 01:51:08.108435 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:08.108410    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-88sfn_d0a185c7-ef98-44ee-8fba-17a3aa80a430/main/0.log"
Jul 14 01:51:08.213441 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:08.213409    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5rjmx_7aa8ce0d-68d0-474a-85fb-2a8d46ff70f1/main/0.log"
Jul 14 01:51:09.846853 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:09.846823    2580 scope.go:117] "RemoveContainer" containerID="49c8e1c5703023bacb721e5aa46a15f73a0e51d873a83ec716b362d0706a0459"
Jul 14 01:51:09.855477 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:09.855460    2580 scope.go:117] "RemoveContainer" containerID="8952f984630bd7fc5a9f3729c734d5984d15eea03cceb1c0036336b2042024bb"
Jul 14 01:51:15.595186 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:15.595158    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lwmb9_6d6a1433-d511-4afe-9860-ca3ca555fe54/global-pull-secret-syncer/0.log"
Jul 14 01:51:15.749416 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:15.749383    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h2hxt_0b619f9b-7574-4199-8d79-01fd455f9829/konnectivity-agent/0.log"
Jul 14 01:51:15.811118 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:15.811090    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-32.ec2.internal_9d3082df98acf7e090e04a68eb61aa6a/haproxy/0.log"
Jul 14 01:51:21.224770 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:21.224737    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-frt9g_fa784b3b-bd5f-4714-8877-816544fdfdaf/kuadrant-console-plugin/0.log"
Jul 14 01:51:23.113882 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.113804    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-wfqhk_239efbb9-39bb-439f-982a-5234ea1e10b5/cluster-monitoring-operator/0.log"
Jul 14 01:51:23.151638 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.151617    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wxxkz_3c1caa54-d144-4f9a-b7b4-9792d5ab544d/kube-state-metrics/0.log"
Jul 14 01:51:23.163307 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.163286    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wxxkz_3c1caa54-d144-4f9a-b7b4-9792d5ab544d/kube-rbac-proxy-main/0.log"
Jul 14 01:51:23.179128 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.179105    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wxxkz_3c1caa54-d144-4f9a-b7b4-9792d5ab544d/kube-rbac-proxy-self/0.log"
Jul 14 01:51:23.201983 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.201957    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86dd9c8cf7-4rj2b_dc45df5b-775b-404e-a9d4-1dec3705980e/metrics-server/0.log"
Jul 14 01:51:23.336732 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.336683    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwt5d_06982999-377f-4b78-a62e-281524f12e4d/node-exporter/0.log"
Jul 14 01:51:23.355975 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.355955    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwt5d_06982999-377f-4b78-a62e-281524f12e4d/kube-rbac-proxy/0.log"
Jul 14 01:51:23.375844 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.375789    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwt5d_06982999-377f-4b78-a62e-281524f12e4d/init-textfile/0.log"
Jul 14 01:51:23.470662 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.470639    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-5rkfp_33876569-f023-4f8f-a8de-c9f65228b8d5/kube-rbac-proxy-main/0.log"
Jul 14 01:51:23.508603 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.508583    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-5rkfp_33876569-f023-4f8f-a8de-c9f65228b8d5/kube-rbac-proxy-self/0.log"
Jul 14 01:51:23.530132 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.530115    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-5rkfp_33876569-f023-4f8f-a8de-c9f65228b8d5/openshift-state-metrics/0.log"
Jul 14 01:51:23.567386 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.567365    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a96771c-e57e-4b42-9b79-e5f6a1adc398/prometheus/0.log"
Jul 14 01:51:23.582783 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.582756    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a96771c-e57e-4b42-9b79-e5f6a1adc398/config-reloader/0.log"
Jul 14 01:51:23.606862 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.606824    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a96771c-e57e-4b42-9b79-e5f6a1adc398/thanos-sidecar/0.log"
Jul 14 01:51:23.620396 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.620379    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a96771c-e57e-4b42-9b79-e5f6a1adc398/kube-rbac-proxy-web/0.log"
Jul 14 01:51:23.639418 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.639347    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a96771c-e57e-4b42-9b79-e5f6a1adc398/kube-rbac-proxy/0.log"
Jul 14 01:51:23.653959 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.653940    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a96771c-e57e-4b42-9b79-e5f6a1adc398/kube-rbac-proxy-thanos/0.log"
Jul 14 01:51:23.672093 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.672076    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8a96771c-e57e-4b42-9b79-e5f6a1adc398/init-config-reloader/0.log"
Jul 14 01:51:23.754135 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.754116    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bcf6b854-58977_a0ed9655-9ae8-49b1-875d-919b17b9c3cf/telemeter-client/0.log"
Jul 14 01:51:23.774340 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.774319    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bcf6b854-58977_a0ed9655-9ae8-49b1-875d-919b17b9c3cf/reload/0.log"
Jul 14 01:51:23.790334 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:23.790315    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bcf6b854-58977_a0ed9655-9ae8-49b1-875d-919b17b9c3cf/kube-rbac-proxy/0.log"
Jul 14 01:51:26.252026 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.251775    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"]
Jul 14 01:51:26.252689 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.252669    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3b8e15-313d-4152-ac86-b367dc26bdb7" containerName="revoke-keys"
Jul 14 01:51:26.252689 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.252692    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3b8e15-313d-4152-ac86-b367dc26bdb7" containerName="revoke-keys"
Jul 14 01:51:26.252847 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.252816    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a3b8e15-313d-4152-ac86-b367dc26bdb7" containerName="revoke-keys"
Jul 14 01:51:26.255954 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.255932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.258037 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.258015    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-njqhj\"/\"kube-root-ca.crt\""
Jul 14 01:51:26.258378 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.258361    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-njqhj\"/\"openshift-service-ca.crt\""
Jul 14 01:51:26.258423 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.258402    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-njqhj\"/\"default-dockercfg-78zpm\""
Jul 14 01:51:26.264736 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.264714    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"]
Jul 14 01:51:26.372058 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.372030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-podres\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.372182 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.372062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8ckg\" (UniqueName: \"kubernetes.io/projected/d69ef609-2a2d-4441-88a1-bcefaf405866-kube-api-access-j8ckg\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.372182 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.372129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-proc\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.372182 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.372154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-sys\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.372182 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.372170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-lib-modules\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473368 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-podres\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473368 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8ckg\" (UniqueName: \"kubernetes.io/projected/d69ef609-2a2d-4441-88a1-bcefaf405866-kube-api-access-j8ckg\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473574 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-proc\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473574 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-sys\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473574 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-lib-modules\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473574 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-podres\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473574 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-proc\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473766 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-sys\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.473766 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.473630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d69ef609-2a2d-4441-88a1-bcefaf405866-lib-modules\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.480312 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.480294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8ckg\" (UniqueName: \"kubernetes.io/projected/d69ef609-2a2d-4441-88a1-bcefaf405866-kube-api-access-j8ckg\") pod \"perf-node-gather-daemonset-m5bfd\" (UID: \"d69ef609-2a2d-4441-88a1-bcefaf405866\") " pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.566370 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.566308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.698139 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.698114    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"]
Jul 14 01:51:26.700303 ip-10-0-141-32 kubenswrapper[2580]: W0714 01:51:26.700264    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd69ef609_2a2d_4441_88a1_bcefaf405866.slice/crio-5d60179fd978d4a9d6f4a429928f552bce20f719140cbbf466823cd06d5ac74d WatchSource:0}: Error finding container 5d60179fd978d4a9d6f4a429928f552bce20f719140cbbf466823cd06d5ac74d: Status 404 returned error can't find the container with id 5d60179fd978d4a9d6f4a429928f552bce20f719140cbbf466823cd06d5ac74d
Jul 14 01:51:26.946935 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.946867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-77k6t_4390937e-579c-4a5e-a8f5-98a19f02b652/volume-data-source-validator/0.log"
Jul 14 01:51:26.981958 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.981930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd" event={"ID":"d69ef609-2a2d-4441-88a1-bcefaf405866","Type":"ContainerStarted","Data":"23d00c0a51175083e3eacacee0f3dead9773ef3b5f8488a1a5fac13a3718818b"}
Jul 14 01:51:26.982082 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.981978    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd" event={"ID":"d69ef609-2a2d-4441-88a1-bcefaf405866","Type":"ContainerStarted","Data":"5d60179fd978d4a9d6f4a429928f552bce20f719140cbbf466823cd06d5ac74d"}
Jul 14 01:51:26.982082 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.982016    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:26.999453 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:26.999411    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd" podStartSLOduration=0.999396239 podStartE2EDuration="999.396239ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:51:26.997562858 +0000 UTC m=+3077.892413497" watchObservedRunningTime="2026-07-14 01:51:26.999396239 +0000 UTC m=+3077.894246879"
Jul 14 01:51:27.911764 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:27.911738    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-64wff_5921a40f-30cb-4ca3-82ec-ce0fd64d96ad/dns/0.log"
Jul 14 01:51:27.926377 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:27.926353    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-64wff_5921a40f-30cb-4ca3-82ec-ce0fd64d96ad/kube-rbac-proxy/0.log"
Jul 14 01:51:28.034613 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:28.034584    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ggkk8_e2073ac1-0532-4124-a3ef-d37024685bc1/dns-node-resolver/0.log"
Jul 14 01:51:28.590157 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:28.590123    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lp74x_20434812-dd9a-43d5-8eff-612bbeff893d/node-ca/0.log"
Jul 14 01:51:29.595035 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:29.594984    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d697d898-xdr7l_72ca077f-314f-4b4d-80ba-d9f7f89fd270/kube-auth-proxy/0.log"
Jul 14 01:51:29.744885 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:29.744853    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ggjh2_4583f204-ce27-431d-8e2a-2fc74b67fd08/payload-processing/0.log"
Jul 14 01:51:30.444064 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:30.444028    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f6wqv_ed40dc6e-0146-4e9f-a1d5-6e92d671ca0f/serve-healthcheck-canary/0.log"
Jul 14 01:51:30.925503 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:30.925471    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-zg4pg_375faa1c-7017-4d2d-bcf4-b9db1a90c6b9/insights-operator/0.log"
Jul 14 01:51:30.926215 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:30.926197    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-zg4pg_375faa1c-7017-4d2d-bcf4-b9db1a90c6b9/insights-operator/1.log"
Jul 14 01:51:31.053950 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:31.053928    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xjcvt_911ff649-c9b4-422e-a473-8213dfead665/kube-rbac-proxy/0.log"
Jul 14 01:51:31.068341 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:31.068317    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xjcvt_911ff649-c9b4-422e-a473-8213dfead665/exporter/0.log"
Jul 14 01:51:31.085707 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:31.085679    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xjcvt_911ff649-c9b4-422e-a473-8213dfead665/extractor/0.log"
Jul 14 01:51:32.998907 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:32.998882    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-njqhj/perf-node-gather-daemonset-m5bfd"
Jul 14 01:51:33.544564 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:33.544509    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75868574bf-zknhx_6dd08212-b3b0-4e6e-b575-b745166eb21e/manager/0.log"
Jul 14 01:51:33.822151 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:33.822070    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc876bb4c-jc6xk_a19ea936-a202-4f6b-8d8d-95d5c1a2bab7/manager/0.log"
Jul 14 01:51:33.834671 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:33.834653    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc876bb4c-jc6xk_a19ea936-a202-4f6b-8d8d-95d5c1a2bab7/copy-manifests/0.log"
Jul 14 01:51:35.091468 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:35.091392    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78f77bcdb4-246v4_89089398-cf9e-4ffd-931e-7e45bf12f4bc/manager/0.log"
Jul 14 01:51:35.135064 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:35.135027    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wdcg7_0b316b17-7a8e-40b4-bec6-6945c5fe4fe7/openshift-lws-operator/0.log"
Jul 14 01:51:36.995119 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:36.995085    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d697d898-xdr7l_72ca077f-314f-4b4d-80ba-d9f7f89fd270/kube-auth-proxy/0.log"
Jul 14 01:51:37.125324 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:37.125295    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ggjh2_4583f204-ce27-431d-8e2a-2fc74b67fd08/payload-processing/0.log"
Jul 14 01:51:38.618826 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:38.618797    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d697d898-xdr7l_72ca077f-314f-4b4d-80ba-d9f7f89fd270/kube-auth-proxy/0.log"
Jul 14 01:51:38.747640 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:38.747612    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ggjh2_4583f204-ce27-431d-8e2a-2fc74b67fd08/payload-processing/0.log"
Jul 14 01:51:40.638184 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:40.638147    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-vfd7q_aaa3ef89-02e7-4724-abf2-cc0c6fe7ebdb/kube-storage-version-migrator-operator/0.log"
Jul 14 01:51:41.890257 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:41.890233    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ksnzm_ccc2deb6-ef0c-4606-9e80-2287b626ee13/kube-multus-additional-cni-plugins/0.log"
Jul 14 01:51:41.907755 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:41.907731    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ksnzm_ccc2deb6-ef0c-4606-9e80-2287b626ee13/egress-router-binary-copy/0.log"
Jul 14 01:51:41.922634 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:41.922614    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ksnzm_ccc2deb6-ef0c-4606-9e80-2287b626ee13/cni-plugins/0.log"
Jul 14 01:51:41.936795 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:41.936776    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ksnzm_ccc2deb6-ef0c-4606-9e80-2287b626ee13/bond-cni-plugin/0.log"
Jul 14 01:51:41.952617 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:41.952597    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ksnzm_ccc2deb6-ef0c-4606-9e80-2287b626ee13/routeoverride-cni/0.log"
Jul 14 01:51:41.967078 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:41.967054    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ksnzm_ccc2deb6-ef0c-4606-9e80-2287b626ee13/whereabouts-cni-bincopy/0.log"
Jul 14 01:51:41.981272 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:41.981253    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ksnzm_ccc2deb6-ef0c-4606-9e80-2287b626ee13/whereabouts-cni/0.log"
Jul 14 01:51:42.018966 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:42.018942    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nx776_628a13aa-d3a0-4ebc-843e-45aafd8338bc/kube-multus/0.log"
Jul 14 01:51:42.148346 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:42.148260    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x2jvq_e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab/network-metrics-daemon/0.log"
Jul 14 01:51:42.161673 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:42.161650    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x2jvq_e14d77c0-4cb8-4bf2-8ed6-f8ecd3e975ab/kube-rbac-proxy/0.log"
Jul 14 01:51:43.273735 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:43.273702    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v5tc5_6569e768-b512-4ed8-8332-69b769ce7f95/ovn-controller/0.log"
Jul 14 01:51:43.286152 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:43.286129    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v5tc5_6569e768-b512-4ed8-8332-69b769ce7f95/ovn-acl-logging/0.log"
Jul 14 01:51:43.308369 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:43.308346    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v5tc5_6569e768-b512-4ed8-8332-69b769ce7f95/kube-rbac-proxy-node/0.log"
Jul 14 01:51:43.328997 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:43.328977    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v5tc5_6569e768-b512-4ed8-8332-69b769ce7f95/kube-rbac-proxy-ovn-metrics/0.log"
Jul 14 01:51:43.343839 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:43.343815    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v5tc5_6569e768-b512-4ed8-8332-69b769ce7f95/northd/0.log"
Jul 14 01:51:43.360830 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:43.360813    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v5tc5_6569e768-b512-4ed8-8332-69b769ce7f95/nbdb/0.log"
Jul 14 01:51:43.375675 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:43.375655    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v5tc5_6569e768-b512-4ed8-8332-69b769ce7f95/sbdb/0.log"
Jul 14 01:51:43.452190 ip-10-0-141-32 kubenswrapper[2580]: I0714 01:51:43.452158    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v5tc5_6569e768-b512-4ed8-8332-69b769ce7f95/ovnkube-controller/0.log"