Jul 02 23:22:33.165627 ip-10-0-137-14 systemd[1]: Starting Kubernetes Kubelet... Jul 02 23:22:33.472375 ip-10-0-137-14 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:22:33.472375 ip-10-0-137-14 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 23:22:33.472375 ip-10-0-137-14 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:22:33.472375 ip-10-0-137-14 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 23:22:33.472375 ip-10-0-137-14 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:22:33.474239 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.474170 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 23:22:33.477996 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.477981 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:33.477996 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.477996 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478000 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478003 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478005 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478008 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478010 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478015 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478019 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478022 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478024 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478027 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478029 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478032 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478034 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478037 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478039 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478047 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478050 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478053 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478055 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:33.478058 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478058 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478061 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478063 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478066 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478069 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478072 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478075 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478077 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478080 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478082 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478085 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478087 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478089 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478092 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478094 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478097 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478099 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478101 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478104 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478106 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:33.478547 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478109 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478111 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478113 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478116 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478118 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478121 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478125 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478128 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478131 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478134 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478137 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478140 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478144 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478146 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478150 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478153 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478155 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478158 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478161 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:33.479024 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478163 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478166 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478168 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478171 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478173 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478176 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478178 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478180 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478182 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478185 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478187 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478189 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478192 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478194 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478197 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478199 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478201 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478204 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478206 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478209 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:33.479533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478211 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478213 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478216 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478218 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478220 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478223 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478225 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478228 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478231 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478233 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478236 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478238 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478241 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478243 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478245 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478248 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478250 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478252 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.478255 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479522 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:33.480009 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479529 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479533 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479535 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479538 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479541 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479543 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479546 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479549 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479552 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479555 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479557 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479560 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479562 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479564 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479567 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479570 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479572 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479575 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479578 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479581 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:33.480500 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479584 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479586 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479589 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479591 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479594 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479596 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479598 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479601 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479606 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479609 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479612 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479615 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479618 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479620 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479623 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479625 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479628 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479630 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479633 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:33.480983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479635 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479638 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479640 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479643 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479645 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479648 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479650 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479653 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479657 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479660 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479663 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479667 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479670 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479672 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479675 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479677 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479680 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479682 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479684 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:33.481455 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479687 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479689 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479692 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479694 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479696 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479699 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479701 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479703 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479706 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479708 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479712 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479714 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479717 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479721 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479724 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479727 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479730 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479732 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479735 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:33.481916 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479737 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479739 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479742 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479744 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479746 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479749 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479752 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479755 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479758 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479760 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479763 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479765 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479768 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479770 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479772 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479775 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479777 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479780 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479782 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479784 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:33.482494 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.479787 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480126 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480130 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480133 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480135 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480138 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480141 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480143 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480146 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480148 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480151 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480153 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480156 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480158 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480161 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480163 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480166 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480168 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480171 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480173 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:33.482983 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480176 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480178 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480181 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480183 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480186 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480188 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480191 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480193 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480195 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480198 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480202 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480206 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480208 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480211 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480214 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480217 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480220 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480222 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480225 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:33.483467 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480228 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480231 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480234 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480238 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480241 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480244 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480248 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480250 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480253 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480256 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480258 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480260 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480263 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480265 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480268 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480270 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480273 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480275 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480278 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480280 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:33.483965 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480282 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480284 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480287 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480289 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480292 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480294 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480296 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480299 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480301 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480303 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480305 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480308 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480310 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480313 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480315 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480317 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480320 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480322 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480325 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480327 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:33.484458 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480329 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480332 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480335 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480337 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480340 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480342 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480345 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480347 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480350 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480354 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480356 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480374 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480376 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480379 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480381 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480384 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480386 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480389 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480393 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:33.484941 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480397 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.480399 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480464 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480470 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480476 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480480 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480484 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480487 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480491 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480494 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480498 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480501 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480504 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480507 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480511 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480514 2555 flags.go:64] FLAG: --cgroup-root="" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480516 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480519 2555 flags.go:64] FLAG: --client-ca-file="" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480522 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480524 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480528 2555 flags.go:64] FLAG: --cluster-domain="" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480531 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480535 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480538 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 23:22:33.485399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480541 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480544 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480547 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480550 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480553 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480555 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480558 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480560 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480563 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480566 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480569 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480572 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480575 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480577 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480580 2555 flags.go:64] FLAG: --enable-server="true" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480583 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480587 2555 flags.go:64] FLAG: --event-burst="100" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480590 2555 flags.go:64] FLAG: --event-qps="50" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480592 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480596 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480598 2555 flags.go:64] FLAG: --eviction-hard="" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480602 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480604 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480607 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480610 2555 flags.go:64] FLAG: --eviction-soft="" Jul 02 23:22:33.485975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480613 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480615 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480618 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480621 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480623 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480630 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480633 2555 flags.go:64] FLAG: --feature-gates="" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480636 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480639 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480642 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480644 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480648 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480650 2555 flags.go:64] FLAG: --help="false" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480653 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480656 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480658 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480661 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480664 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480667 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480670 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480673 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480675 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480678 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 23:22:33.486593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480680 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480683 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480686 2555 flags.go:64] FLAG: --kube-reserved="" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480688 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480703 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480706 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480709 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480712 2555 flags.go:64] FLAG: --lock-file="" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480715 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480717 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480720 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480729 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480732 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480735 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480738 2555 flags.go:64] FLAG: --logging-format="text" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480741 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480744 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480747 2555 flags.go:64] FLAG: --manifest-url="" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480749 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480753 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480756 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480759 2555 flags.go:64] FLAG: --max-pods="110" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480762 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480765 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480767 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 23:22:33.487177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480770 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480773 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480776 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480778 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480784 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480787 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480790 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480793 2555 flags.go:64] FLAG: --pod-cidr="" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480795 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480800 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480802 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480805 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480816 2555 flags.go:64] FLAG: --port="10250" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480819 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480822 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e03abbc98c9c171a" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480825 2555 flags.go:64] FLAG: --qos-reserved="" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480827 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480830 2555 flags.go:64] FLAG: --register-node="true" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480833 2555 flags.go:64] FLAG: --register-with-taints="" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480836 2555 flags.go:64] FLAG: --registry-burst="10" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480838 2555 flags.go:64] FLAG: --registry-qps="5" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480842 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480845 2555 flags.go:64] FLAG: --reserved-memory="" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480848 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480851 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 23:22:33.487823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480853 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480856 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480859 2555 flags.go:64] FLAG: --runonce="false" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480861 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480864 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480867 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480870 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480872 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480875 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480878 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480880 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480883 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480885 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480888 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480890 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480893 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480896 2555 flags.go:64] FLAG: --system-cgroups="" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480898 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480903 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480906 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480908 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480914 2555 flags.go:64] FLAG: --tls-min-version="" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480916 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480919 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480921 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 23:22:33.488438 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480924 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480927 2555 flags.go:64] FLAG: --v="2" Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480930 2555 flags.go:64] FLAG: --version="false" Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480935 2555 flags.go:64] FLAG: --vmodule="" Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480938 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.480941 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481044 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481048 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481051 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481054 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481057 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481060 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481063 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481065 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481068 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481071 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481074 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481076 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481079 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481082 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481085 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:33.489035 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481087 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481089 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481092 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481094 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481097 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481100 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481102 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481110 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481113 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481116 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481118 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481122 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481125 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481128 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481131 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481134 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481137 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481139 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481143 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:33.489548 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481146 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481149 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481152 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481154 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481158 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481162 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481165 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481167 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481170 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481173 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481175 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481178 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481181 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481184 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481186 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481188 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481191 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481193 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481195 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481198 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:33.490031 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481201 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481205 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481208 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481210 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481213 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481215 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481217 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481221 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481224 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481226 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481228 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481231 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481233 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481236 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481238 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481241 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481243 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481245 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481248 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481250 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:33.490550 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481253 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481256 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481258 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481260 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481263 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481265 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481267 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481270 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481272 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481274 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481277 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481279 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481283 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481286 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481289 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481291 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481294 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481296 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481299 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481303 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:33.491023 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481305 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481307 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481310 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481312 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.481314 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.481319 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.487393 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.487406 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.487425 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.487433 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487493 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487498 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487502 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487505 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487507 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:33.491530 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487510 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487513 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487515 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487518 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487520 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487523 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487525 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487528 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487530 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487533 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487535 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487538 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487540 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487543 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487545 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487547 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487550 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487552 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487555 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487557 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:33.491900 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487559 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487562 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487564 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487567 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487573 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487577 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487580 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487583 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487585 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487588 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487591 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487593 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487596 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487598 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487601 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487603 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487605 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487608 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487610 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487613 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:33.492403 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487615 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487618 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487620 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487622 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487625 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487627 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487629 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487632 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487635 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487637 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487640 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487642 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487645 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487647 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487650 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487653 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487656 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487659 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487661 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487664 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:33.492918 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487666 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487669 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487671 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487674 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487676 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487678 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487681 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487683 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487685 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487688 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487690 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487693 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487696 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487701 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487704 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487707 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487709 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487712 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487715 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:33.493427 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487717 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487720 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487722 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487725 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487728 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487730 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487733 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487735 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487738 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487740 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487743 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487746 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487748 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487751 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487753 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:33.493895 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.487758 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487862 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487867 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487870 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487872 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487875 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487878 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487881 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487883 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487886 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487888 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487891 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487893 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487896 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487898 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487901 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487903 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487906 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487909 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487911 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:33.494269 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487914 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487916 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487918 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487921 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487923 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487926 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487928 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487931 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487933 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487936 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487939 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487941 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487944 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487946 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487948 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487950 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487953 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487955 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487958 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487961 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:33.494828 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487964 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487967 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487969 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487972 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487974 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487977 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487979 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487981 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487984 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487986 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487989 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487991 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487993 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487996 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.487998 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488000 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488002 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488005 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488008 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:33.495310 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488010 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488012 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488015 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488017 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488020 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488022 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488024 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488027 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488029 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488032 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488034 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488036 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488038 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488041 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488043 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488045 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488048 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488052 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488055 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488057 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:33.495793 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488060 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488063 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488066 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488068 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488071 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488073 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488075 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488077 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488080 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488082 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488085 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488089 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488092 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488095 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488097 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488100 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488106 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488109 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488111 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488114 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:33.496258 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:33.488116 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:33.496784 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.488121 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:22:33.496784 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.488209 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 23:22:33.496784 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.490709 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 23:22:33.496784 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.491846 2555 server.go:1019] "Starting client certificate rotation" Jul 02 23:22:33.496784 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.491940 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:22:33.496784 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.491979 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:22:33.496784 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.494165 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:22:33.499243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.499223 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:22:33.508339 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.508322 2555 log.go:25] "Validated CRI v1 runtime API" Jul 02 23:22:33.510144 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.510128 2555 log.go:25] "Validated CRI v1 image API" Jul 02 23:22:33.510912 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.510899 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 23:22:33.513877 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.513851 2555 fs.go:135] Filesystem UUIDs: map[1e368274-5318-4910-ac4e-5f9c0da2398a:/dev/nvme0n1p4 46b643ec-4917-4e27-9a6c-e22a1b6db814:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 02 23:22:33.513948 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.513877 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 23:22:33.518641 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.518540 2555 manager.go:216] Machine: {Timestamp:2026-07-02 23:22:33.517349942 +0000 UTC m=+0.256642390 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100809 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c62496ed292618d58071e262663a6 SystemUUID:ec2c6249-6ed2-9261-8d58-071e262663a6 BootID:f8634360-deb3-46ea-a638-920a23d643f7 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:93:24:6f:9b:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:93:24:6f:9b:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:91:f3:6c:4e:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 23:22:33.518641 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.518633 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 23:22:33.518759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.518699 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 23:22:33.521200 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.521174 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 23:22:33.521323 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.521203 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-14.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 23:22:33.521841 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.521831 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 23:22:33.521873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.521844 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 23:22:33.521873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.521851 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:22:33.521873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.521865 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:22:33.522088 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.522073 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:22:33.523025 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.523015 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 23:22:33.526596 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.526585 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:22:33.526701 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.526693 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 23:22:33.526736 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.526722 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 02 23:22:33.526736 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.526732 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 23:22:33.526785 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.526743 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 23:22:33.526785 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.526753 2555 kubelet.go:398] "Adding apiserver pod source" Jul 02 23:22:33.526785 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.526761 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 23:22:33.527055 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.527043 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:22:33.527091 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.527060 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:22:33.528696 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.528680 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 23:22:33.530643 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.530629 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 23:22:33.530700 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.530658 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 23:22:33.531667 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531655 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 23:22:33.531714 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531672 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 23:22:33.531714 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531678 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 23:22:33.531714 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531684 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 23:22:33.531714 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531695 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 23:22:33.531714 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531703 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 23:22:33.531714 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531713 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 23:22:33.531873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531718 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 23:22:33.531873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531724 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 23:22:33.531873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531740 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 23:22:33.531873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531756 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 23:22:33.531873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531762 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 23:22:33.531873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531783 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 23:22:33.531873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.531788 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 23:22:33.535034 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.535020 2555 server.go:1268] "Started kubelet" Jul 02 23:22:33.535158 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.535126 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 23:22:33.535215 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.535148 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 23:22:33.535277 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.535236 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 23:22:33.535941 ip-10-0-137-14 systemd[1]: Started Kubernetes Kubelet. Jul 02 23:22:33.538132 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.538106 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 23:22:33.538132 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.538122 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-14.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 23:22:33.538427 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.538398 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 23:22:33.538427 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.538398 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 23:22:33.540306 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.540291 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 02 23:22:33.541520 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.541503 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w6hg8" Jul 02 23:22:33.543146 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.543125 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 23:22:33.544040 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544026 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 23:22:33.544040 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544034 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 23:22:33.544164 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.544102 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:33.544561 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544542 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 23:22:33.544561 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544550 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 23:22:33.544739 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544582 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 23:22:33.544739 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544686 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 02 23:22:33.544739 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544694 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 02 23:22:33.544915 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544758 2555 factory.go:55] Registering systemd factory Jul 02 23:22:33.544915 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.544788 2555 factory.go:223] Registration of the systemd container factory successfully Jul 02 23:22:33.545070 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.543537 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-14.ec2.internal.18be9c7ac1b35936 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-14.ec2.internal,UID:ip-10-0-137-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-14.ec2.internal,},FirstTimestamp:2026-07-02 23:22:33.534994742 +0000 UTC m=+0.274287191,LastTimestamp:2026-07-02 23:22:33.534994742 +0000 UTC m=+0.274287191,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-14.ec2.internal,}" Jul 02 23:22:33.545177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.545161 2555 factory.go:153] Registering CRI-O factory Jul 02 23:22:33.545177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.545177 2555 factory.go:223] Registration of the crio container factory successfully Jul 02 23:22:33.545277 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.545220 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 23:22:33.545277 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.545237 2555 factory.go:103] Registering Raw factory Jul 02 23:22:33.545277 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.545247 2555 manager.go:1196] Started watching for new ooms in manager Jul 02 23:22:33.545699 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.545685 2555 manager.go:349] Starting recovery of all containers Jul 02 23:22:33.548342 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.548317 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 23:22:33.548461 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.548441 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 23:22:33.549489 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.549470 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w6hg8" Jul 02 23:22:33.554959 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.554814 2555 manager.go:354] Recovery completed Jul 02 23:22:33.559681 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.559668 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:33.561204 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.561191 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:33.561257 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.561218 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:33.561257 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.561229 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:33.561757 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.561743 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 23:22:33.561757 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.561758 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 23:22:33.561844 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.561776 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:33.561844 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.561777 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:22:33.562545 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.562479 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:33.562545 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.562507 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:33.562545 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.562517 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:33.563458 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.563307 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-14.ec2.internal.18be9c7ac34349bb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-14.ec2.internal,UID:ip-10-0-137-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-14.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-14.ec2.internal,},FirstTimestamp:2026-07-02 23:22:33.561205179 +0000 UTC m=+0.300497631,LastTimestamp:2026-07-02 23:22:33.561205179 +0000 UTC m=+0.300497631,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-14.ec2.internal,}" Jul 02 23:22:33.564795 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.564721 2555 policy_none.go:49] "None policy: Start" Jul 02 23:22:33.564869 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.564799 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 23:22:33.564869 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.564815 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 23:22:33.566853 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.566839 2555 policy_none.go:47] "Start" Jul 02 23:22:33.606923 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.606906 2555 manager.go:337] "Starting Device Plugin manager" Jul 02 23:22:33.606995 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.606940 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 23:22:33.606995 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.606950 2555 server.go:87] "Starting device plugin registration server" Jul 02 23:22:33.607179 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.607167 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 23:22:33.607232 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.607181 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 23:22:33.607286 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.607251 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 23:22:33.607377 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.607351 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 23:22:33.607439 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.607381 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 23:22:33.607759 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.607740 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 23:22:33.607829 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.607781 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:33.674959 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.674939 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 23:22:33.677074 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.676231 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 23:22:33.677074 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.676260 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 23:22:33.677074 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.676276 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 23:22:33.677074 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.676302 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 23:22:33.678646 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.678629 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:33.707884 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.707867 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:33.709701 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.709687 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:33.709755 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.709712 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:33.709755 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.709725 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:33.709755 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.709744 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.717231 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.717218 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.717279 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.717236 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-14.ec2.internal\": node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:33.730217 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.730174 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:33.777310 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.777288 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal"] Jul 02 23:22:33.777409 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.777341 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:33.778082 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.778069 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:33.778127 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.778094 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:33.778127 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.778107 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:33.780299 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.780286 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:33.780458 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.780443 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.780502 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.780473 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:33.780962 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.780946 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:33.781035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.780977 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:33.781035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.780990 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:33.781035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.780993 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:33.781035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.781012 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:33.781035 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.781022 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:33.783139 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.783124 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.783192 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.783154 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:33.783771 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.783757 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:33.783857 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.783778 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:33.783857 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.783792 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:33.805892 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.805875 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-14.ec2.internal\" not found" node="ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.810151 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.810134 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-14.ec2.internal\" not found" node="ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.831126 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.831109 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:33.846899 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.846885 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d7cc513f8160a936a236def8c0790b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal\" (UID: \"f7d7cc513f8160a936a236def8c0790b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.846969 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.846907 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/394edcdb51bee6f5516971692e04a798-config\") pod \"kube-apiserver-proxy-ip-10-0-137-14.ec2.internal\" (UID: \"394edcdb51bee6f5516971692e04a798\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.846969 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.846922 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7d7cc513f8160a936a236def8c0790b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal\" (UID: \"f7d7cc513f8160a936a236def8c0790b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.931300 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:33.931285 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:33.947694 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.947671 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7d7cc513f8160a936a236def8c0790b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal\" (UID: \"f7d7cc513f8160a936a236def8c0790b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.947772 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.947696 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d7cc513f8160a936a236def8c0790b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal\" (UID: \"f7d7cc513f8160a936a236def8c0790b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.947772 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.947712 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/394edcdb51bee6f5516971692e04a798-config\") pod \"kube-apiserver-proxy-ip-10-0-137-14.ec2.internal\" (UID: \"394edcdb51bee6f5516971692e04a798\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.947842 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.947768 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7d7cc513f8160a936a236def8c0790b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal\" (UID: \"f7d7cc513f8160a936a236def8c0790b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.947842 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.947757 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/394edcdb51bee6f5516971692e04a798-config\") pod \"kube-apiserver-proxy-ip-10-0-137-14.ec2.internal\" (UID: \"394edcdb51bee6f5516971692e04a798\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" Jul 02 23:22:33.947842 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:33.947783 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d7cc513f8160a936a236def8c0790b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal\" (UID: \"f7d7cc513f8160a936a236def8c0790b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:34.032112 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.032069 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:34.108519 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.108498 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:34.113104 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.113091 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" Jul 02 23:22:34.132999 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.132966 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:34.233054 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.233031 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:34.333499 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.333482 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:34.433940 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.433923 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-14.ec2.internal\" not found" Jul 02 23:22:34.444685 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.444664 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:34.453009 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.452989 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:34.491016 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.490995 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 23:22:34.491518 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.491149 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:22:34.491518 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.491158 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:22:34.491518 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.491180 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:22:34.516027 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.516005 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:34.526911 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.526890 2555 apiserver.go:52] "Watching apiserver" Jul 02 23:22:34.535382 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.535362 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 23:22:34.535672 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.535652 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-qrhzq","openshift-multus/multus-nlbp4","kube-system/konnectivity-agent-kknfx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7","openshift-cluster-node-tuning-operator/tuned-nbfwk","openshift-multus/network-metrics-daemon-88h28","openshift-network-diagnostics/network-check-target-6qsmv","openshift-network-operator/iptables-alerter-d5jhn","openshift-ovn-kubernetes/ovnkube-node-8mhj9","kube-system/global-pull-secret-syncer-hhcrv","openshift-dns/node-resolver-qpvc2","openshift-image-registry/node-ca-pnb5p"] Jul 02 23:22:34.538268 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.538239 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.540283 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.540264 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xdq4h\"" Jul 02 23:22:34.540631 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.540607 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 23:22:34.540723 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.540620 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.540788 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.540650 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 23:22:34.540788 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.540727 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 23:22:34.540896 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.540801 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 23:22:34.542997 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.542977 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.543140 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.543110 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ftggf\"" Jul 02 23:22:34.543250 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.543229 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 23:22:34.543580 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.543561 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 23:22:34.543650 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.543605 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:22:34.544499 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.544479 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" Jul 02 23:22:34.544580 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.544523 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 23:22:34.545206 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.545154 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 23:22:34.545326 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.545275 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:34.545457 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.545410 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mjhb9\"" Jul 02 23:22:34.545647 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.545627 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 23:22:34.547326 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.547310 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 23:22:34.547743 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.547723 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.548027 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.547924 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rbs4z\"" Jul 02 23:22:34.548027 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.547965 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 23:22:34.549758 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.549737 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g47tw\"" Jul 02 23:22:34.549878 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.549845 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67cf61d5-5693-41f5-8555-de6b76bd89a5-konnectivity-ca\") pod \"konnectivity-agent-kknfx\" (UID: \"67cf61d5-5693-41f5-8555-de6b76bd89a5\") " pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:34.549964 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.549883 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-netns\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.549964 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.549915 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.549964 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.549943 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67cf61d5-5693-41f5-8555-de6b76bd89a5-agent-certs\") pod \"konnectivity-agent-kknfx\" (UID: \"67cf61d5-5693-41f5-8555-de6b76bd89a5\") " pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:34.550124 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.549969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-kubelet\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550124 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.549997 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l849n\" (UniqueName: \"kubernetes.io/projected/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-kube-api-access-l849n\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.550124 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550025 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-os-release\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.550124 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550078 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.550124 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550100 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 23:22:34.550389 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550176 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 23:22:34.550389 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550100 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 23:22:34.550389 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550110 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-cni-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550389 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550254 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.550389 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550298 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7bdff9fb-c404-400f-9db5-c432287ed695-multus-daemon-config\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550389 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550324 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-os-release\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550389 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550350 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-socket-dir-parent\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550376 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94qk9\" (UniqueName: \"kubernetes.io/projected/7bdff9fb-c404-400f-9db5-c432287ed695-kube-api-access-94qk9\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550475 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-host-slash\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550500 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550526 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-cnibin\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550544 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-iptables-alerter-script\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550559 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bdff9fb-c404-400f-9db5-c432287ed695-cni-binary-copy\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550575 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-k8s-cni-cncf-io\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-cni-multus\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550627 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-system-cni-dir\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550658 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550709 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-cni-bin\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550757 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-conf-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.550793 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550784 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-multus-certs\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.551359 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550809 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-etc-kubernetes\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.551359 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550834 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-system-cni-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.551359 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550858 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-hostroot\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.551359 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550889 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-cnibin\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.551359 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.550919 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9f85\" (UniqueName: \"kubernetes.io/projected/507bb56b-96aa-4ab3-b284-b589f3959f4e-kube-api-access-d9f85\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.552664 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.552649 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:34.552761 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.552702 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4rgtx\"" Jul 02 23:22:34.552821 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.552776 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:22:34.552821 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.552777 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:34.552948 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.552930 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 23:22:34.554547 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.554510 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 23:17:33 +0000 UTC" deadline="2027-12-16 00:10:29.043453032 +0000 UTC" Jul 02 23:22:34.554624 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.554547 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12744h47m54.488908651s" Jul 02 23:22:34.555012 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.554987 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:34.555066 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.555048 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:34.555949 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.555934 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:22:34.557223 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.557207 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" Jul 02 23:22:34.557466 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.557448 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.560862 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.560838 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 23:22:34.560975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.560912 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 23:22:34.561277 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.561259 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 23:22:34.561394 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.561365 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 23:22:34.561443 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.561409 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gbb7s\"" Jul 02 23:22:34.561855 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.561825 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 23:22:34.561943 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.561886 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 23:22:34.562205 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.562188 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.562294 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.562275 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:34.563069 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.563046 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:22:34.564870 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.564855 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.567220 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.567197 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 23:22:34.567220 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.567215 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 23:22:34.567394 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.567248 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal"] Jul 02 23:22:34.567394 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.567341 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.567394 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.567357 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mfnms\"" Jul 02 23:22:34.567556 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.567417 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal"] Jul 02 23:22:34.567556 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.567496 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:22:34.569597 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.569574 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 23:22:34.569688 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.569635 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 23:22:34.569688 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.569671 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qksbw\"" Jul 02 23:22:34.569816 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.569796 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 23:22:34.582980 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.582962 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5pvvb" Jul 02 23:22:34.590391 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.590373 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5pvvb" Jul 02 23:22:34.645351 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.645337 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 23:22:34.651086 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651064 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysconfig\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.651174 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651106 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:34.651174 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651135 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-etc-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.651174 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651159 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovnkube-config\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.651309 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651191 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bdff9fb-c404-400f-9db5-c432287ed695-cni-binary-copy\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651309 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-k8s-cni-cncf-io\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651309 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651242 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-cni-multus\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651309 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-system-cni-dir\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.651309 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651292 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-kubelet-config\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.651309 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651305 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-k8s-cni-cncf-io\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651317 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-kubernetes\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651340 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-run\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651342 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-cni-multus\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651365 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-conf-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651360 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-system-cni-dir\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651451 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-conf-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxjsq\" (UniqueName: \"kubernetes.io/projected/2dfdc72a-9586-4e0b-b68d-7008640b7770-kube-api-access-vxjsq\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651522 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-registration-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651611 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-slash\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651629 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-hostroot\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651646 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67cf61d5-5693-41f5-8555-de6b76bd89a5-konnectivity-ca\") pod \"konnectivity-agent-kknfx\" (UID: \"67cf61d5-5693-41f5-8555-de6b76bd89a5\") " pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651661 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-dbus\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651690 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651715 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2dfdc72a-9586-4e0b-b68d-7008640b7770-hosts-file\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651718 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-hostroot\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651736 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzfhh\" (UniqueName: \"kubernetes.io/projected/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-kube-api-access-wzfhh\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:34.651759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651754 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-cni-netd\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-netns\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651822 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651824 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-netns\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651858 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67cf61d5-5693-41f5-8555-de6b76bd89a5-agent-certs\") pod \"konnectivity-agent-kknfx\" (UID: \"67cf61d5-5693-41f5-8555-de6b76bd89a5\") " pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651867 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bdff9fb-c404-400f-9db5-c432287ed695-cni-binary-copy\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651895 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-modprobe-d\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651928 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-etc-selinux\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.651979 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-sys-fs\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652018 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652049 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-kubelet\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652076 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l849n\" (UniqueName: \"kubernetes.io/projected/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-kube-api-access-l849n\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652087 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-kubelet\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652105 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-os-release\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652133 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652161 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/401fe81a-d58d-430b-bb94-f73fdce48999-host\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652187 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-sys\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.652623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652190 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-os-release\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652282 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652316 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-var-lib-kubelet\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652284 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67cf61d5-5693-41f5-8555-de6b76bd89a5-konnectivity-ca\") pod \"konnectivity-agent-kknfx\" (UID: \"67cf61d5-5693-41f5-8555-de6b76bd89a5\") " pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7bdff9fb-c404-400f-9db5-c432287ed695-multus-daemon-config\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652513 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652632 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-tuned\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652661 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3760c5ab-6c71-4178-b673-5b9a82aa48b9-tmp\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652680 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-systemd-units\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652696 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-run-netns\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652721 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovn-node-metrics-cert\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652828 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-os-release\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652911 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-os-release\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652957 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-socket-dir-parent\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652945 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.652985 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653013 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czzxb\" (UniqueName: \"kubernetes.io/projected/401fe81a-d58d-430b-bb94-f73fdce48999-kube-api-access-czzxb\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.653476 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653041 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlrpl\" (UniqueName: \"kubernetes.io/projected/48b6b582-61e5-477b-9de4-4156a1368a90-kube-api-access-rlrpl\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653064 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-cni-bin\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653064 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7bdff9fb-c404-400f-9db5-c432287ed695-multus-daemon-config\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653105 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-socket-dir-parent\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653117 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-cnibin\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653176 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-iptables-alerter-script\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653190 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-cnibin\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653244 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysctl-conf\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653277 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-host\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653301 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653330 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-node-log\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653582 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/507bb56b-96aa-4ab3-b284-b589f3959f4e-cni-binary-copy\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653630 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovnkube-script-lib\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653711 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653734 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-cni-bin\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653765 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-multus-certs\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-etc-kubernetes\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.654243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-systemd\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653856 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-run-multus-certs\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653862 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-iptables-alerter-script\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653873 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-host-var-lib-cni-bin\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653904 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-socket-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653926 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-etc-kubernetes\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653950 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-system-cni-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.653991 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-cnibin\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654446 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9f85\" (UniqueName: \"kubernetes.io/projected/507bb56b-96aa-4ab3-b284-b589f3959f4e-kube-api-access-d9f85\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654501 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/401fe81a-d58d-430b-bb94-f73fdce48999-serviceca\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654538 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp4ds\" (UniqueName: \"kubernetes.io/projected/3760c5ab-6c71-4178-b673-5b9a82aa48b9-kube-api-access-mp4ds\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654583 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654606 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-system-cni-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654630 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-lib-modules\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654678 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-device-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654689 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-cnibin\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654709 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-ovn\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.655134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654741 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dfdc72a-9586-4e0b-b68d-7008640b7770-tmp-dir\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654778 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysctl-d\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654832 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654866 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-cni-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654899 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-systemd\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655130 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bdff9fb-c404-400f-9db5-c432287ed695-multus-cni-dir\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655176 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/507bb56b-96aa-4ab3-b284-b589f3959f4e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.654933 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-log-socket\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655408 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-env-overrides\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655457 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94qk9\" (UniqueName: \"kubernetes.io/projected/7bdff9fb-c404-400f-9db5-c432287ed695-kube-api-access-94qk9\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655481 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-host-slash\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655501 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-kubelet\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655534 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-var-lib-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655567 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgxps\" (UniqueName: \"kubernetes.io/projected/c4086f1f-8cb6-4adc-9364-62e754a6259a-kube-api-access-vgxps\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.655999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.655882 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-host-slash\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.656822 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.656792 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67cf61d5-5693-41f5-8555-de6b76bd89a5-agent-certs\") pod \"konnectivity-agent-kknfx\" (UID: \"67cf61d5-5693-41f5-8555-de6b76bd89a5\") " pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:34.661582 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.661563 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l849n\" (UniqueName: \"kubernetes.io/projected/20870e68-d6f9-4d98-8d6a-7add9ef18ed1-kube-api-access-l849n\") pod \"iptables-alerter-d5jhn\" (UID: \"20870e68-d6f9-4d98-8d6a-7add9ef18ed1\") " pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.662570 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.662552 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94qk9\" (UniqueName: \"kubernetes.io/projected/7bdff9fb-c404-400f-9db5-c432287ed695-kube-api-access-94qk9\") pod \"multus-nlbp4\" (UID: \"7bdff9fb-c404-400f-9db5-c432287ed695\") " pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.663162 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.663135 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9f85\" (UniqueName: \"kubernetes.io/projected/507bb56b-96aa-4ab3-b284-b589f3959f4e-kube-api-access-d9f85\") pod \"multus-additional-cni-plugins-qrhzq\" (UID: \"507bb56b-96aa-4ab3-b284-b589f3959f4e\") " pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.681520 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.681480 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d7cc513f8160a936a236def8c0790b.slice/crio-cd81f4dbabac72ef4051273e0245189e0110e29ceac265a5f88d905e5df4ba13 WatchSource:0}: Error finding container cd81f4dbabac72ef4051273e0245189e0110e29ceac265a5f88d905e5df4ba13: Status 404 returned error can't find the container with id cd81f4dbabac72ef4051273e0245189e0110e29ceac265a5f88d905e5df4ba13 Jul 02 23:22:34.682239 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.682221 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394edcdb51bee6f5516971692e04a798.slice/crio-ea935a996f5371607731da54754dd982aeb8825c86a7070f9ada7efa5b8d8866 WatchSource:0}: Error finding container ea935a996f5371607731da54754dd982aeb8825c86a7070f9ada7efa5b8d8866: Status 404 returned error can't find the container with id ea935a996f5371607731da54754dd982aeb8825c86a7070f9ada7efa5b8d8866 Jul 02 23:22:34.685932 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.685910 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:22:34.756433 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756405 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/401fe81a-d58d-430b-bb94-f73fdce48999-host\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.756531 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756435 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-sys\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756531 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756452 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-var-lib-kubelet\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756531 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756467 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-tuned\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756531 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756469 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/401fe81a-d58d-430b-bb94-f73fdce48999-host\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.756531 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756480 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3760c5ab-6c71-4178-b673-5b9a82aa48b9-tmp\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756531 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756496 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-systemd-units\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756531 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756530 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-sys\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756545 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-run-netns\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756562 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-var-lib-kubelet\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756572 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-systemd-units\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756565 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-run-netns\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756581 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovn-node-metrics-cert\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756608 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czzxb\" (UniqueName: \"kubernetes.io/projected/401fe81a-d58d-430b-bb94-f73fdce48999-kube-api-access-czzxb\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlrpl\" (UniqueName: \"kubernetes.io/projected/48b6b582-61e5-477b-9de4-4156a1368a90-kube-api-access-rlrpl\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756640 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-cni-bin\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756663 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysctl-conf\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756684 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-cni-bin\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756685 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-host\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756723 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756729 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-host\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756746 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-node-log\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756769 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovnkube-script-lib\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756774 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756795 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-systemd\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.756903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756817 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-node-log\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756822 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-socket-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756827 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysctl-conf\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756859 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/401fe81a-d58d-430b-bb94-f73fdce48999-serviceca\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756888 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp4ds\" (UniqueName: \"kubernetes.io/projected/3760c5ab-6c71-4178-b673-5b9a82aa48b9-kube-api-access-mp4ds\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756916 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756942 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-lib-modules\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756966 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-device-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756999 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-socket-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756890 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-systemd\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757051 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-ovn\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757068 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757176 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-lib-modules\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757223 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-device-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.756990 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-ovn\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757264 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dfdc72a-9586-4e0b-b68d-7008640b7770-tmp-dir\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757289 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysctl-d\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.758063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757314 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757328 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/401fe81a-d58d-430b-bb94-f73fdce48999-serviceca\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757338 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757364 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-systemd\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757387 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-log-socket\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757435 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-env-overrides\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757460 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysctl-d\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757462 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-kubelet\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757498 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-var-lib-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757505 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-kubelet\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757526 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgxps\" (UniqueName: \"kubernetes.io/projected/c4086f1f-8cb6-4adc-9364-62e754a6259a-kube-api-access-vgxps\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757551 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysconfig\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.757578 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757580 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757609 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-etc-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovnkube-config\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.757671 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs podName:f4cf293e-362f-45b5-aad9-e3a34a8f52f8 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:35.257628548 +0000 UTC m=+1.996920992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs") pod "network-metrics-daemon-88h28" (UID: "f4cf293e-362f-45b5-aad9-e3a34a8f52f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:34.758958 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757704 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-kubelet-config\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-kubernetes\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757740 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dfdc72a-9586-4e0b-b68d-7008640b7770-tmp-dir\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757755 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-run\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757782 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxjsq\" (UniqueName: \"kubernetes.io/projected/2dfdc72a-9586-4e0b-b68d-7008640b7770-kube-api-access-vxjsq\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757807 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-registration-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757833 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-slash\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757860 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-dbus\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757889 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757917 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2dfdc72a-9586-4e0b-b68d-7008640b7770-hosts-file\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757943 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzfhh\" (UniqueName: \"kubernetes.io/projected/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-kube-api-access-wzfhh\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.757988 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-cni-netd\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758017 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-modprobe-d\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758044 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-etc-selinux\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758067 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovnkube-script-lib\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758076 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-sys-fs\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758104 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.759587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758169 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovnkube-config\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758181 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758199 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c4086f1f-8cb6-4adc-9364-62e754a6259a-env-overrides\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758208 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-dbus\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758224 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-var-lib-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.758287 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758302 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-run-systemd\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758287 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-log-socket\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.758342 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret podName:c1b6e71e-c320-46f2-b9f7-07fd0a76cabc nodeName:}" failed. No retries permitted until 2026-07-02 23:22:35.258323894 +0000 UTC m=+1.997616335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret") pod "global-pull-secret-syncer-hhcrv" (UID: "c1b6e71e-c320-46f2-b9f7-07fd0a76cabc") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758379 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-cni-netd\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758404 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-etc-openvswitch\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758427 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758458 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-kubelet-config\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758495 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-sysconfig\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758496 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-sys-fs\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758520 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-kubernetes\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758556 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-run\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.760105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758568 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c4086f1f-8cb6-4adc-9364-62e754a6259a-host-slash\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.760665 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758622 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-modprobe-d\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.760665 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758648 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2dfdc72a-9586-4e0b-b68d-7008640b7770-hosts-file\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.760665 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758648 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-etc-selinux\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.760665 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758798 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48b6b582-61e5-477b-9de4-4156a1368a90-registration-dir\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.760665 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.758957 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3760c5ab-6c71-4178-b673-5b9a82aa48b9-tmp\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.760665 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.759152 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3760c5ab-6c71-4178-b673-5b9a82aa48b9-etc-tuned\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.760665 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.759388 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c4086f1f-8cb6-4adc-9364-62e754a6259a-ovn-node-metrics-cert\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.764667 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.764640 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czzxb\" (UniqueName: \"kubernetes.io/projected/401fe81a-d58d-430b-bb94-f73fdce48999-kube-api-access-czzxb\") pod \"node-ca-pnb5p\" (UID: \"401fe81a-d58d-430b-bb94-f73fdce48999\") " pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.764913 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.764893 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp4ds\" (UniqueName: \"kubernetes.io/projected/3760c5ab-6c71-4178-b673-5b9a82aa48b9-kube-api-access-mp4ds\") pod \"tuned-nbfwk\" (UID: \"3760c5ab-6c71-4178-b673-5b9a82aa48b9\") " pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.766573 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.766555 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:34.766663 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.766578 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:34.766663 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.766588 2555 projected.go:196] Error preparing data for projected volume kube-api-access-d6s9d for pod openshift-network-diagnostics/network-check-target-6qsmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:34.766663 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:34.766641 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d podName:11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb nodeName:}" failed. No retries permitted until 2026-07-02 23:22:35.266623739 +0000 UTC m=+2.005916192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d6s9d" (UniqueName: "kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d") pod "network-check-target-6qsmv" (UID: "11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:34.768857 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.768831 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxjsq\" (UniqueName: \"kubernetes.io/projected/2dfdc72a-9586-4e0b-b68d-7008640b7770-kube-api-access-vxjsq\") pod \"node-resolver-qpvc2\" (UID: \"2dfdc72a-9586-4e0b-b68d-7008640b7770\") " pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.769321 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.769296 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzfhh\" (UniqueName: \"kubernetes.io/projected/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-kube-api-access-wzfhh\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:34.769510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.769494 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlrpl\" (UniqueName: \"kubernetes.io/projected/48b6b582-61e5-477b-9de4-4156a1368a90-kube-api-access-rlrpl\") pod \"aws-ebs-csi-driver-node-j4zb7\" (UID: \"48b6b582-61e5-477b-9de4-4156a1368a90\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.769736 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.769720 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgxps\" (UniqueName: \"kubernetes.io/projected/c4086f1f-8cb6-4adc-9364-62e754a6259a-kube-api-access-vgxps\") pod \"ovnkube-node-8mhj9\" (UID: \"c4086f1f-8cb6-4adc-9364-62e754a6259a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.863144 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.863097 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nlbp4" Jul 02 23:22:34.872896 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.872878 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bdff9fb_c404_400f_9db5_c432287ed695.slice/crio-d3063ee449158671e1b04c8dcfb03e9dfaa50a9dac1b138cbe7e94b4e13d829c WatchSource:0}: Error finding container d3063ee449158671e1b04c8dcfb03e9dfaa50a9dac1b138cbe7e94b4e13d829c: Status 404 returned error can't find the container with id d3063ee449158671e1b04c8dcfb03e9dfaa50a9dac1b138cbe7e94b4e13d829c Jul 02 23:22:34.883975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.883958 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d5jhn" Jul 02 23:22:34.893674 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.893656 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20870e68_d6f9_4d98_8d6a_7add9ef18ed1.slice/crio-19be74cd64d744db634470fb5395b171ff599fcc91c64dd3577a4a44906ae010 WatchSource:0}: Error finding container 19be74cd64d744db634470fb5395b171ff599fcc91c64dd3577a4a44906ae010: Status 404 returned error can't find the container with id 19be74cd64d744db634470fb5395b171ff599fcc91c64dd3577a4a44906ae010 Jul 02 23:22:34.894667 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.894649 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" Jul 02 23:22:34.899288 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.899266 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:34.903557 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.903540 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507bb56b_96aa_4ab3_b284_b589f3959f4e.slice/crio-5a4d5b67d02aeecfed63d8c202e457e7ad32faa084473cf26c659114e7914182 WatchSource:0}: Error finding container 5a4d5b67d02aeecfed63d8c202e457e7ad32faa084473cf26c659114e7914182: Status 404 returned error can't find the container with id 5a4d5b67d02aeecfed63d8c202e457e7ad32faa084473cf26c659114e7914182 Jul 02 23:22:34.907945 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.907929 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67cf61d5_5693_41f5_8555_de6b76bd89a5.slice/crio-3e1558ec298f8ff73c6aa003a2ec17f52221d6ccc50cfc9d19f71a66533604ef WatchSource:0}: Error finding container 3e1558ec298f8ff73c6aa003a2ec17f52221d6ccc50cfc9d19f71a66533604ef: Status 404 returned error can't find the container with id 3e1558ec298f8ff73c6aa003a2ec17f52221d6ccc50cfc9d19f71a66533604ef Jul 02 23:22:34.915496 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.915476 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" Jul 02 23:22:34.924402 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.924379 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b6b582_61e5_477b_9de4_4156a1368a90.slice/crio-87d5fff80aac9f629c403394890c438cbb4b55d276bf45f6b9840c781646220c WatchSource:0}: Error finding container 87d5fff80aac9f629c403394890c438cbb4b55d276bf45f6b9840c781646220c: Status 404 returned error can't find the container with id 87d5fff80aac9f629c403394890c438cbb4b55d276bf45f6b9840c781646220c Jul 02 23:22:34.932462 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.932445 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" Jul 02 23:22:34.940449 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.940415 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3760c5ab_6c71_4178_b673_5b9a82aa48b9.slice/crio-823f01549cbbaf684ba708503860b33460d418438a5c6181b603a5e83cc58741 WatchSource:0}: Error finding container 823f01549cbbaf684ba708503860b33460d418438a5c6181b603a5e83cc58741: Status 404 returned error can't find the container with id 823f01549cbbaf684ba708503860b33460d418438a5c6181b603a5e83cc58741 Jul 02 23:22:34.954300 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.954280 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:22:34.963381 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.963358 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4086f1f_8cb6_4adc_9364_62e754a6259a.slice/crio-edd158e733d3e70b7169e496a224652ff3bfc1ea1748630c0d24c4c49271ac5b WatchSource:0}: Error finding container edd158e733d3e70b7169e496a224652ff3bfc1ea1748630c0d24c4c49271ac5b: Status 404 returned error can't find the container with id edd158e733d3e70b7169e496a224652ff3bfc1ea1748630c0d24c4c49271ac5b Jul 02 23:22:34.979508 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.979489 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qpvc2" Jul 02 23:22:34.985750 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:34.985734 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pnb5p" Jul 02 23:22:34.987564 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.987526 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dfdc72a_9586_4e0b_b68d_7008640b7770.slice/crio-9d0286cee0244a01b1ac8ce7b6fa167c90b663262feffbc87d1880a6eecabb85 WatchSource:0}: Error finding container 9d0286cee0244a01b1ac8ce7b6fa167c90b663262feffbc87d1880a6eecabb85: Status 404 returned error can't find the container with id 9d0286cee0244a01b1ac8ce7b6fa167c90b663262feffbc87d1880a6eecabb85 Jul 02 23:22:34.996344 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:22:34.996325 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401fe81a_d58d_430b_bb94_f73fdce48999.slice/crio-645393ae0dd5907c8358d212a0c5f5685ab5b54daa05c89e802227c9f5d08880 WatchSource:0}: Error finding container 645393ae0dd5907c8358d212a0c5f5685ab5b54daa05c89e802227c9f5d08880: Status 404 returned error can't find the container with id 645393ae0dd5907c8358d212a0c5f5685ab5b54daa05c89e802227c9f5d08880 Jul 02 23:22:35.262099 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.262014 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:35.262099 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.262084 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:35.262305 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:35.262221 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:35.262305 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:35.262283 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret podName:c1b6e71e-c320-46f2-b9f7-07fd0a76cabc nodeName:}" failed. No retries permitted until 2026-07-02 23:22:36.262262283 +0000 UTC m=+3.001554736 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret") pod "global-pull-secret-syncer-hhcrv" (UID: "c1b6e71e-c320-46f2-b9f7-07fd0a76cabc") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:35.262744 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:35.262723 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:35.262856 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:35.262780 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs podName:f4cf293e-362f-45b5-aad9-e3a34a8f52f8 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:36.262763093 +0000 UTC m=+3.002055533 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs") pod "network-metrics-daemon-88h28" (UID: "f4cf293e-362f-45b5-aad9-e3a34a8f52f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:35.297709 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.297679 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:35.362828 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.362744 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:35.363097 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:35.363069 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:35.363097 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:35.363092 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:35.363254 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:35.363106 2555 projected.go:196] Error preparing data for projected volume kube-api-access-d6s9d for pod openshift-network-diagnostics/network-check-target-6qsmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:35.363254 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:35.363161 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d podName:11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb nodeName:}" failed. No retries permitted until 2026-07-02 23:22:36.363141369 +0000 UTC m=+3.102433822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d6s9d" (UniqueName: "kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d") pod "network-check-target-6qsmv" (UID: "11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:35.593437 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.593300 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:17:34 +0000 UTC" deadline="2028-01-02 07:35:25.105671579 +0000 UTC" Jul 02 23:22:35.593437 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.593358 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13160h12m49.512318082s" Jul 02 23:22:35.699861 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.699737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pnb5p" event={"ID":"401fe81a-d58d-430b-bb94-f73fdce48999","Type":"ContainerStarted","Data":"645393ae0dd5907c8358d212a0c5f5685ab5b54daa05c89e802227c9f5d08880"} Jul 02 23:22:35.708256 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.708227 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" event={"ID":"3760c5ab-6c71-4178-b673-5b9a82aa48b9","Type":"ContainerStarted","Data":"823f01549cbbaf684ba708503860b33460d418438a5c6181b603a5e83cc58741"} Jul 02 23:22:35.723408 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.723375 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" event={"ID":"48b6b582-61e5-477b-9de4-4156a1368a90","Type":"ContainerStarted","Data":"87d5fff80aac9f629c403394890c438cbb4b55d276bf45f6b9840c781646220c"} Jul 02 23:22:35.743424 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.743399 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kknfx" event={"ID":"67cf61d5-5693-41f5-8555-de6b76bd89a5","Type":"ContainerStarted","Data":"3e1558ec298f8ff73c6aa003a2ec17f52221d6ccc50cfc9d19f71a66533604ef"} Jul 02 23:22:35.761163 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.761135 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerStarted","Data":"5a4d5b67d02aeecfed63d8c202e457e7ad32faa084473cf26c659114e7914182"} Jul 02 23:22:35.783014 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.782983 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d5jhn" event={"ID":"20870e68-d6f9-4d98-8d6a-7add9ef18ed1","Type":"ContainerStarted","Data":"19be74cd64d744db634470fb5395b171ff599fcc91c64dd3577a4a44906ae010"} Jul 02 23:22:35.810890 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.810842 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nlbp4" event={"ID":"7bdff9fb-c404-400f-9db5-c432287ed695","Type":"ContainerStarted","Data":"d3063ee449158671e1b04c8dcfb03e9dfaa50a9dac1b138cbe7e94b4e13d829c"} Jul 02 23:22:35.821556 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.821194 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" event={"ID":"394edcdb51bee6f5516971692e04a798","Type":"ContainerStarted","Data":"ea935a996f5371607731da54754dd982aeb8825c86a7070f9ada7efa5b8d8866"} Jul 02 23:22:35.842054 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.842026 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qpvc2" event={"ID":"2dfdc72a-9586-4e0b-b68d-7008640b7770","Type":"ContainerStarted","Data":"9d0286cee0244a01b1ac8ce7b6fa167c90b663262feffbc87d1880a6eecabb85"} Jul 02 23:22:35.847352 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.847287 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"edd158e733d3e70b7169e496a224652ff3bfc1ea1748630c0d24c4c49271ac5b"} Jul 02 23:22:35.870955 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:35.870926 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" event={"ID":"f7d7cc513f8160a936a236def8c0790b","Type":"ContainerStarted","Data":"cd81f4dbabac72ef4051273e0245189e0110e29ceac265a5f88d905e5df4ba13"} Jul 02 23:22:36.271028 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:36.270944 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:36.271028 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:36.271011 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:36.271249 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.271126 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:36.271249 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.271185 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret podName:c1b6e71e-c320-46f2-b9f7-07fd0a76cabc nodeName:}" failed. No retries permitted until 2026-07-02 23:22:38.271167242 +0000 UTC m=+5.010459682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret") pod "global-pull-secret-syncer-hhcrv" (UID: "c1b6e71e-c320-46f2-b9f7-07fd0a76cabc") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:36.271716 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.271631 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:36.271844 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.271755 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs podName:f4cf293e-362f-45b5-aad9-e3a34a8f52f8 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:38.271736991 +0000 UTC m=+5.011029430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs") pod "network-metrics-daemon-88h28" (UID: "f4cf293e-362f-45b5-aad9-e3a34a8f52f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:36.372012 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:36.371982 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:36.372694 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.372157 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:36.372694 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.372177 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:36.372694 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.372194 2555 projected.go:196] Error preparing data for projected volume kube-api-access-d6s9d for pod openshift-network-diagnostics/network-check-target-6qsmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:36.372694 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.372248 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d podName:11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb nodeName:}" failed. No retries permitted until 2026-07-02 23:22:38.372230821 +0000 UTC m=+5.111523259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d6s9d" (UniqueName: "kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d") pod "network-check-target-6qsmv" (UID: "11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:36.594941 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:36.594897 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:17:34 +0000 UTC" deadline="2028-01-16 16:20:43.731083693 +0000 UTC" Jul 02 23:22:36.594941 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:36.594939 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13504h58m7.136150295s" Jul 02 23:22:36.678346 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:36.677307 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:36.678346 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.677654 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:36.678346 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:36.678012 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:36.678346 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.678111 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:36.678346 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:36.678201 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:36.678346 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:36.678296 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:37.205478 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:37.205426 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:37.613230 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:37.613199 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:38.295014 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:38.294209 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:38.295014 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:38.294284 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:38.295014 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.294449 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:38.295014 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.294509 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret podName:c1b6e71e-c320-46f2-b9f7-07fd0a76cabc nodeName:}" failed. No retries permitted until 2026-07-02 23:22:42.294491709 +0000 UTC m=+9.033784148 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret") pod "global-pull-secret-syncer-hhcrv" (UID: "c1b6e71e-c320-46f2-b9f7-07fd0a76cabc") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:38.295014 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.294905 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:38.295014 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.294955 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs podName:f4cf293e-362f-45b5-aad9-e3a34a8f52f8 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:42.294937497 +0000 UTC m=+9.034229937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs") pod "network-metrics-daemon-88h28" (UID: "f4cf293e-362f-45b5-aad9-e3a34a8f52f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:38.395467 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:38.394844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:38.395467 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.395042 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:38.395467 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.395063 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:38.395467 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.395075 2555 projected.go:196] Error preparing data for projected volume kube-api-access-d6s9d for pod openshift-network-diagnostics/network-check-target-6qsmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:38.395467 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.395128 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d podName:11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb nodeName:}" failed. No retries permitted until 2026-07-02 23:22:42.395111066 +0000 UTC m=+9.134403505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d6s9d" (UniqueName: "kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d") pod "network-check-target-6qsmv" (UID: "11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:38.677061 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:38.676988 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:38.677511 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.677104 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:38.677511 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:38.677459 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:38.677606 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.677535 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:38.677606 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:38.677597 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:38.677702 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:38.677677 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:40.677408 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:40.677375 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:40.677408 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:40.677410 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:40.677886 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:40.677529 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:40.678069 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:40.677922 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:40.678069 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:40.677976 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:40.678069 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:40.678036 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:42.333305 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:42.333265 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:42.333750 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:42.333348 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:42.333750 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.333504 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:42.333750 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.333566 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret podName:c1b6e71e-c320-46f2-b9f7-07fd0a76cabc nodeName:}" failed. No retries permitted until 2026-07-02 23:22:50.333548724 +0000 UTC m=+17.072841162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret") pod "global-pull-secret-syncer-hhcrv" (UID: "c1b6e71e-c320-46f2-b9f7-07fd0a76cabc") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:42.333975 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.333960 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:42.334023 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.334013 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs podName:f4cf293e-362f-45b5-aad9-e3a34a8f52f8 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:50.333997826 +0000 UTC m=+17.073290267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs") pod "network-metrics-daemon-88h28" (UID: "f4cf293e-362f-45b5-aad9-e3a34a8f52f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:42.433833 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:42.433798 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:42.433987 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.433946 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:42.433987 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.433963 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:42.433987 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.433973 2555 projected.go:196] Error preparing data for projected volume kube-api-access-d6s9d for pod openshift-network-diagnostics/network-check-target-6qsmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:42.434104 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.434023 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d podName:11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb nodeName:}" failed. No retries permitted until 2026-07-02 23:22:50.434009568 +0000 UTC m=+17.173302004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d6s9d" (UniqueName: "kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d") pod "network-check-target-6qsmv" (UID: "11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:42.676649 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:42.676515 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:42.676649 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:42.676540 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:42.676649 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.676632 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:42.677083 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:42.676915 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:42.677083 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.676969 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:42.677083 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:42.677040 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:44.677114 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:44.677029 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:44.677574 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:44.677148 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:44.677574 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:44.677464 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:44.677574 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:44.677466 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:44.677574 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:44.677567 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:44.677767 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:44.677652 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:46.676682 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:46.676607 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:46.676682 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:46.676649 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:46.677142 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:46.676720 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:46.677142 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:46.676737 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:46.677142 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:46.676801 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:46.677142 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:46.676866 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:48.677305 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:48.677124 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:48.677305 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:48.677237 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:48.677305 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:48.677248 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:48.677305 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:48.677254 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:48.677873 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:48.677346 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:48.677873 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:48.677455 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:50.400143 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:50.400107 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:50.400686 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:50.400173 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:50.400686 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.400250 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:50.400686 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.400280 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:50.400686 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.400309 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs podName:f4cf293e-362f-45b5-aad9-e3a34a8f52f8 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:06.400294885 +0000 UTC m=+33.139587321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs") pod "network-metrics-daemon-88h28" (UID: "f4cf293e-362f-45b5-aad9-e3a34a8f52f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:50.400686 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.400333 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret podName:c1b6e71e-c320-46f2-b9f7-07fd0a76cabc nodeName:}" failed. No retries permitted until 2026-07-02 23:23:06.40031695 +0000 UTC m=+33.139609385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret") pod "global-pull-secret-syncer-hhcrv" (UID: "c1b6e71e-c320-46f2-b9f7-07fd0a76cabc") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:50.501147 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:50.501117 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:50.501281 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.501270 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:50.501322 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.501287 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:50.501322 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.501297 2555 projected.go:196] Error preparing data for projected volume kube-api-access-d6s9d for pod openshift-network-diagnostics/network-check-target-6qsmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:50.501419 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.501348 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d podName:11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb nodeName:}" failed. No retries permitted until 2026-07-02 23:23:06.50133573 +0000 UTC m=+33.240628167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d6s9d" (UniqueName: "kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d") pod "network-check-target-6qsmv" (UID: "11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:50.677540 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:50.677481 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:50.677693 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:50.677478 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:50.677693 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.677588 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:50.677693 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.677676 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:50.677859 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:50.677481 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:50.677859 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:50.677778 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:52.676899 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:52.676871 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:52.677466 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:52.676983 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:52.677466 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:52.676995 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:52.677466 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:52.677017 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:52.677466 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:52.677114 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:52.677466 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:52.677194 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:54.676487 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:54.676450 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:54.676951 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:54.676497 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:54.676951 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:54.676571 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:54.676951 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:54.676458 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:54.676951 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:54.676705 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:54.676951 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:54.676816 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:55.924453 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.924208 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nlbp4" event={"ID":"7bdff9fb-c404-400f-9db5-c432287ed695","Type":"ContainerStarted","Data":"c225944c1df2ccece6636394e3b324337c2d66aeacc0821d4648dc436a350d65"} Jul 02 23:22:55.926556 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.926218 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" event={"ID":"394edcdb51bee6f5516971692e04a798","Type":"ContainerStarted","Data":"63c84eedb558c0ba9bcf266b3ebb7c7cab780c017f64924219537ab5ffc0527d"} Jul 02 23:22:55.929348 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.929331 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:22:55.929676 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.929658 2555 generic.go:356] "Generic (PLEG): container finished" podID="c4086f1f-8cb6-4adc-9364-62e754a6259a" containerID="e52c2ee9f2a98b1e692375b4f02afea05bde0bce6c891c56db1d2e8b7b7aca25" exitCode=1 Jul 02 23:22:55.929737 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.929715 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"4d80e97351de987ec03b44c6d3e7ce0e1a503a7fcb2fc0203ef0976262f17b54"} Jul 02 23:22:55.929778 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.929760 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"b68b2e74e0531f85d551e1fa01e316c0279d8e0ae67586a6239374fb3d25f61e"} Jul 02 23:22:55.929831 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.929788 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"584c8b7d581b356e08ca548f9ea7a238a2c00329f5bab11f4effcf5728f106a0"} Jul 02 23:22:55.929831 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.929814 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerDied","Data":"e52c2ee9f2a98b1e692375b4f02afea05bde0bce6c891c56db1d2e8b7b7aca25"} Jul 02 23:22:55.929906 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.929845 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"4115894ce35eeb99f6fa0b82e954c8a05eccfe8605d310a924dd9d31220c8cd0"} Jul 02 23:22:55.931696 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.931677 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" event={"ID":"3760c5ab-6c71-4178-b673-5b9a82aa48b9","Type":"ContainerStarted","Data":"68ab35dc66dfc6ef61945b11c8aa7d815ccbe6224466b660b3b079af3f586f7c"} Jul 02 23:22:55.939311 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.939223 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nlbp4" podStartSLOduration=2.737092757 podStartE2EDuration="22.939198785s" totalImagesPullingTime="20.202106028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:55.93890613 +0000 UTC m=+22.678198589" watchObservedRunningTime="2026-07-02 23:22:55.939198785 +0000 UTC m=+22.678491245" Jul 02 23:22:55.950889 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.950853 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-14.ec2.internal" podStartSLOduration=21.950841778 podStartE2EDuration="21.950841778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:55.950589315 +0000 UTC m=+22.689881774" watchObservedRunningTime="2026-07-02 23:22:55.950841778 +0000 UTC m=+22.690134236" Jul 02 23:22:55.964639 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:55.964607 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nbfwk" podStartSLOduration=1.829956006 podStartE2EDuration="21.964600633s" totalImagesPullingTime="20.134644627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:55.964429696 +0000 UTC m=+22.703722157" watchObservedRunningTime="2026-07-02 23:22:55.964600633 +0000 UTC m=+22.703893090" Jul 02 23:22:56.677228 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.677064 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:56.677353 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.677103 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:56.677440 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.677102 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:56.677491 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:56.677291 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:56.677491 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:56.677386 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:56.677491 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:56.677472 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:56.935842 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.935812 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7d7cc513f8160a936a236def8c0790b" containerID="0e445d98fcdf846faad08a1d52cf13d47dbab8a52e86b3fcb12ef9085b6257c2" exitCode=0 Jul 02 23:22:56.936283 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.935902 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" event={"ID":"f7d7cc513f8160a936a236def8c0790b","Type":"ContainerDied","Data":"0e445d98fcdf846faad08a1d52cf13d47dbab8a52e86b3fcb12ef9085b6257c2"} Jul 02 23:22:56.938179 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.938141 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pnb5p" event={"ID":"401fe81a-d58d-430b-bb94-f73fdce48999","Type":"ContainerStarted","Data":"23d201ef1c39d749918e8fa4e5e84cf96b8eebbb9169144ef6620b046650083e"} Jul 02 23:22:56.939917 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.939897 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" event={"ID":"48b6b582-61e5-477b-9de4-4156a1368a90","Type":"ContainerStarted","Data":"7a5898db0e5dd583b92e9b4773a2643ce23d5ebf8446862c32d22fe564f9af3d"} Jul 02 23:22:56.941462 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.941387 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kknfx" event={"ID":"67cf61d5-5693-41f5-8555-de6b76bd89a5","Type":"ContainerStarted","Data":"de150c6dc93a4987f4ec9fed486d16175e89187fbe21694a05d437663e45509b"} Jul 02 23:22:56.942977 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.942957 2555 generic.go:356] "Generic (PLEG): container finished" podID="507bb56b-96aa-4ab3-b284-b589f3959f4e" containerID="d81604e17abce05beec0960197d0a95ce5cc99d31ef046ab671a8c92f68a0eef" exitCode=0 Jul 02 23:22:56.943081 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.943027 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerDied","Data":"d81604e17abce05beec0960197d0a95ce5cc99d31ef046ab671a8c92f68a0eef"} Jul 02 23:22:56.945298 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.945276 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d5jhn" event={"ID":"20870e68-d6f9-4d98-8d6a-7add9ef18ed1","Type":"ContainerStarted","Data":"a69dcb98a67b7aab58be580299bfb0a6ce3134d8120df82d177847aaf1714a4e"} Jul 02 23:22:56.946965 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.946945 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qpvc2" event={"ID":"2dfdc72a-9586-4e0b-b68d-7008640b7770","Type":"ContainerStarted","Data":"022a502cd3878d3c74e725418131bb09454ffd34ca42a0384fb13efa1bc96a32"} Jul 02 23:22:56.950098 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.950078 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:22:56.950486 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.950468 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"8418be8c2692acabe9c1703724c1ab0044044e82dc432497d57266e03d202dd7"} Jul 02 23:22:56.984914 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:56.984866 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kknfx" podStartSLOduration=2.980596566 podStartE2EDuration="22.984851783s" totalImagesPullingTime="20.004255217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:56.962150493 +0000 UTC m=+23.701442952" watchObservedRunningTime="2026-07-02 23:22:56.984851783 +0000 UTC m=+23.724144244" Jul 02 23:22:57.010584 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.010551 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pnb5p" podStartSLOduration=3.083159305 podStartE2EDuration="23.010539748s" totalImagesPullingTime="19.927380443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:56.998494723 +0000 UTC m=+23.737787201" watchObservedRunningTime="2026-07-02 23:22:57.010539748 +0000 UTC m=+23.749832205" Jul 02 23:22:57.010902 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.010883 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d5jhn" podStartSLOduration=3.992516092 podStartE2EDuration="24.010877182s" totalImagesPullingTime="20.01836109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:57.010413716 +0000 UTC m=+23.749706174" watchObservedRunningTime="2026-07-02 23:22:57.010877182 +0000 UTC m=+23.750169639" Jul 02 23:22:57.025340 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.025299 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qpvc2" podStartSLOduration=2.959697659 podStartE2EDuration="23.025286984s" totalImagesPullingTime="20.065589325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:57.024502532 +0000 UTC m=+23.763794991" watchObservedRunningTime="2026-07-02 23:22:57.025286984 +0000 UTC m=+23.764579443" Jul 02 23:22:57.560117 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.559795 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 23:22:57.620479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.620395 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T23:22:57.559815838Z","UUID":"834506b3-3b56-450b-baa1-97ed57a87ac1","Handler":null,"Name":"","Endpoint":""} Jul 02 23:22:57.621756 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.621740 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 23:22:57.621877 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.621780 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 23:22:57.954594 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.954563 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" event={"ID":"48b6b582-61e5-477b-9de4-4156a1368a90","Type":"ContainerStarted","Data":"3741be7ab384922ca4a1948d74372693d76411e5398001aa22367b2105f48b45"} Jul 02 23:22:57.956335 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.956306 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" event={"ID":"f7d7cc513f8160a936a236def8c0790b","Type":"ContainerStarted","Data":"1663aa1ceb22eca8d11dab070f7cb771e627630fc49017835bd5a96579ee60bd"} Jul 02 23:22:57.970919 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:57.970878 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-14.ec2.internal" podStartSLOduration=23.970866003 podStartE2EDuration="23.970866003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:57.970278521 +0000 UTC m=+24.709570984" watchObservedRunningTime="2026-07-02 23:22:57.970866003 +0000 UTC m=+24.710158460" Jul 02 23:22:58.676666 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.676634 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:22:58.676937 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.676635 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:22:58.676937 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:58.676740 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:22:58.676937 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:58.676843 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:22:58.676937 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.676645 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:22:58.677141 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:22:58.676943 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:22:58.781564 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.781538 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:58.782235 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.782212 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:58.962022 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.961963 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:22:58.962421 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.962390 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"b5f8d57c385c420cf7e7477ba4a54e098a902409f5500ed1c602e816af4a8e7a"} Jul 02 23:22:58.964611 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.964584 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" event={"ID":"48b6b582-61e5-477b-9de4-4156a1368a90","Type":"ContainerStarted","Data":"c056d150b5dd2475f30956f763b14b48736fe60e40309a011823217f13478f23"} Jul 02 23:22:58.965197 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.965144 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:58.965321 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.965304 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kknfx" Jul 02 23:22:58.992916 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:22:58.992876 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j4zb7" podStartSLOduration=2.577490371 podStartE2EDuration="24.992864621s" totalImagesPullingTime="22.41537425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:58.979913028 +0000 UTC m=+25.719205486" watchObservedRunningTime="2026-07-02 23:22:58.992864621 +0000 UTC m=+25.732157079" Jul 02 23:23:00.676988 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:00.676786 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:00.677509 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:00.676787 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:00.677509 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:00.677083 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:23:00.677509 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:00.676814 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:00.677509 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:00.677195 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:23:00.677509 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:00.677306 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:23:01.973510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:01.973299 2555 generic.go:356] "Generic (PLEG): container finished" podID="507bb56b-96aa-4ab3-b284-b589f3959f4e" containerID="4315d5ac3cdf0ea58e70fb03732322e11da82fc2748ff4a6e3b3f7e131e5c37f" exitCode=0 Jul 02 23:23:01.974163 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:01.973390 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerDied","Data":"4315d5ac3cdf0ea58e70fb03732322e11da82fc2748ff4a6e3b3f7e131e5c37f"} Jul 02 23:23:01.977072 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:01.977048 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:23:01.977437 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:01.977416 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"19714c022197df9ad5e369e500e8e0b6def11eeadc0a58f414f9b80ef9a03898"} Jul 02 23:23:01.977688 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:01.977672 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:23:01.977773 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:01.977717 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:23:01.977827 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:01.977769 2555 scope.go:117] "RemoveContainer" containerID="e52c2ee9f2a98b1e692375b4f02afea05bde0bce6c891c56db1d2e8b7b7aca25" Jul 02 23:23:01.993884 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:01.993867 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:23:02.677203 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:02.677177 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:02.677446 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:02.677177 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:02.677446 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:02.677276 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:23:02.677446 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:02.677176 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:02.677446 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:02.677407 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:23:02.677592 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:02.677456 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:23:02.983805 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:02.983781 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:23:02.984162 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:02.984137 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" event={"ID":"c4086f1f-8cb6-4adc-9364-62e754a6259a","Type":"ContainerStarted","Data":"9c2733dba5e4774a14d180af5f25e94531e69400845bc1c31ae816b1a2b27fd3"} Jul 02 23:23:02.984397 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:02.984376 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:23:02.999721 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:02.999696 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:23:03.009906 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.009870 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" podStartSLOduration=8.667039255 podStartE2EDuration="29.009856775s" totalImagesPullingTime="20.34281752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:03.008301016 +0000 UTC m=+29.747593475" watchObservedRunningTime="2026-07-02 23:23:03.009856775 +0000 UTC m=+29.749149233" Jul 02 23:23:03.088600 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.088578 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6qsmv"] Jul 02 23:23:03.088709 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.088671 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:03.088768 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:03.088752 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:23:03.091122 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.091096 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hhcrv"] Jul 02 23:23:03.091228 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.091188 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:03.091301 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:03.091281 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:23:03.091631 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.091609 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-88h28"] Jul 02 23:23:03.091719 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.091706 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:03.091806 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:03.091789 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:23:03.987473 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.987437 2555 generic.go:356] "Generic (PLEG): container finished" podID="507bb56b-96aa-4ab3-b284-b589f3959f4e" containerID="ef4a074130f1f42c33dd616abd855bcfc6871eb7b3ad8631ecb7b0b073efc694" exitCode=0 Jul 02 23:23:03.987836 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:03.987520 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerDied","Data":"ef4a074130f1f42c33dd616abd855bcfc6871eb7b3ad8631ecb7b0b073efc694"} Jul 02 23:23:04.677056 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:04.676846 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:04.677210 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:04.676869 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:04.677210 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:04.677157 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:23:04.677320 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:04.676917 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:04.677320 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:04.677217 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:23:04.677440 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:04.677334 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:23:05.993821 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:05.993796 2555 generic.go:356] "Generic (PLEG): container finished" podID="507bb56b-96aa-4ab3-b284-b589f3959f4e" containerID="b9d248400bec39dec7062cbded60f65ddb5e55f2e187b7cfa70b077e84cb4d55" exitCode=0 Jul 02 23:23:05.994239 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:05.993833 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerDied","Data":"b9d248400bec39dec7062cbded60f65ddb5e55f2e187b7cfa70b077e84cb4d55"} Jul 02 23:23:06.425022 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:06.424993 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:06.425146 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:06.425049 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:06.425146 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.425134 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:23:06.425274 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.425145 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:23:06.425274 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.425194 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs podName:f4cf293e-362f-45b5-aad9-e3a34a8f52f8 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:38.425177566 +0000 UTC m=+65.164470002 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs") pod "network-metrics-daemon-88h28" (UID: "f4cf293e-362f-45b5-aad9-e3a34a8f52f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:23:06.425274 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.425209 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret podName:c1b6e71e-c320-46f2-b9f7-07fd0a76cabc nodeName:}" failed. No retries permitted until 2026-07-02 23:23:38.425203393 +0000 UTC m=+65.164495828 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret") pod "global-pull-secret-syncer-hhcrv" (UID: "c1b6e71e-c320-46f2-b9f7-07fd0a76cabc") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:23:06.525541 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:06.525508 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:06.525669 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.525632 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:23:06.525669 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.525652 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:23:06.525669 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.525663 2555 projected.go:196] Error preparing data for projected volume kube-api-access-d6s9d for pod openshift-network-diagnostics/network-check-target-6qsmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:23:06.525799 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.525718 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d podName:11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb nodeName:}" failed. No retries permitted until 2026-07-02 23:23:38.525701991 +0000 UTC m=+65.264994429 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d6s9d" (UniqueName: "kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d") pod "network-check-target-6qsmv" (UID: "11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:23:06.676910 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:06.676845 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:06.677033 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:06.676952 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:06.677033 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.676957 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:23:06.677033 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:06.676970 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:06.677187 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.677058 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:23:06.677187 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:06.677140 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:23:08.677226 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:08.676745 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:08.677226 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:08.676869 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:08.677226 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:08.676877 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhcrv" podUID="c1b6e71e-c320-46f2-b9f7-07fd0a76cabc" Jul 02 23:23:08.677226 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:08.676907 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:08.677226 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:08.676984 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qsmv" podUID="11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb" Jul 02 23:23:08.677226 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:08.677063 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88h28" podUID="f4cf293e-362f-45b5-aad9-e3a34a8f52f8" Jul 02 23:23:09.134981 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.134952 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-14.ec2.internal" event="NodeReady" Jul 02 23:23:09.135136 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.135064 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 23:23:09.174742 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.174711 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cgggg"] Jul 02 23:23:09.177596 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.177572 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbp86"] Jul 02 23:23:09.177714 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.177612 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.181513 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.180614 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qpvbp\"" Jul 02 23:23:09.181513 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.180747 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 23:23:09.181513 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.180855 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 23:23:09.182170 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.182152 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbp86" Jul 02 23:23:09.184511 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.184221 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 23:23:09.184511 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.184422 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 23:23:09.184652 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.184619 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 23:23:09.184852 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.184837 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-24d9j\"" Jul 02 23:23:09.185946 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.185927 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cgggg"] Jul 02 23:23:09.188893 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.188873 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbp86"] Jul 02 23:23:09.351019 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.350988 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c931cb-95f1-4dc4-8730-204905465bcf-config-volume\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.351173 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.351029 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2ddb\" (UniqueName: \"kubernetes.io/projected/a3c931cb-95f1-4dc4-8730-204905465bcf-kube-api-access-j2ddb\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.351173 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.351110 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.351283 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.351179 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkrpr\" (UniqueName: \"kubernetes.io/projected/070538f1-749e-45d0-bf8d-f3219633dd6d-kube-api-access-zkrpr\") pod \"ingress-canary-gbp86\" (UID: \"070538f1-749e-45d0-bf8d-f3219633dd6d\") " pod="openshift-ingress-canary/ingress-canary-gbp86" Jul 02 23:23:09.351283 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.351231 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c931cb-95f1-4dc4-8730-204905465bcf-tmp-dir\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.351283 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.351259 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/070538f1-749e-45d0-bf8d-f3219633dd6d-cert\") pod \"ingress-canary-gbp86\" (UID: \"070538f1-749e-45d0-bf8d-f3219633dd6d\") " pod="openshift-ingress-canary/ingress-canary-gbp86" Jul 02 23:23:09.452275 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.452197 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/070538f1-749e-45d0-bf8d-f3219633dd6d-cert\") pod \"ingress-canary-gbp86\" (UID: \"070538f1-749e-45d0-bf8d-f3219633dd6d\") " pod="openshift-ingress-canary/ingress-canary-gbp86" Jul 02 23:23:09.452275 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.452255 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c931cb-95f1-4dc4-8730-204905465bcf-config-volume\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.452275 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.452285 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2ddb\" (UniqueName: \"kubernetes.io/projected/a3c931cb-95f1-4dc4-8730-204905465bcf-kube-api-access-j2ddb\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.452554 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.452332 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.452554 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.452396 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkrpr\" (UniqueName: \"kubernetes.io/projected/070538f1-749e-45d0-bf8d-f3219633dd6d-kube-api-access-zkrpr\") pod \"ingress-canary-gbp86\" (UID: \"070538f1-749e-45d0-bf8d-f3219633dd6d\") " pod="openshift-ingress-canary/ingress-canary-gbp86" Jul 02 23:23:09.452554 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.452435 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c931cb-95f1-4dc4-8730-204905465bcf-tmp-dir\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.452786 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.452765 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c931cb-95f1-4dc4-8730-204905465bcf-tmp-dir\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.452913 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:09.452892 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:09.453011 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.452913 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c931cb-95f1-4dc4-8730-204905465bcf-config-volume\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.453011 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:09.452948 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls podName:a3c931cb-95f1-4dc4-8730-204905465bcf nodeName:}" failed. No retries permitted until 2026-07-02 23:23:09.952930819 +0000 UTC m=+36.692223271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls") pod "dns-default-cgggg" (UID: "a3c931cb-95f1-4dc4-8730-204905465bcf") : secret "dns-default-metrics-tls" not found Jul 02 23:23:09.457266 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.457239 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/070538f1-749e-45d0-bf8d-f3219633dd6d-cert\") pod \"ingress-canary-gbp86\" (UID: \"070538f1-749e-45d0-bf8d-f3219633dd6d\") " pod="openshift-ingress-canary/ingress-canary-gbp86" Jul 02 23:23:09.463759 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.463714 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkrpr\" (UniqueName: \"kubernetes.io/projected/070538f1-749e-45d0-bf8d-f3219633dd6d-kube-api-access-zkrpr\") pod \"ingress-canary-gbp86\" (UID: \"070538f1-749e-45d0-bf8d-f3219633dd6d\") " pod="openshift-ingress-canary/ingress-canary-gbp86" Jul 02 23:23:09.463845 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.463805 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2ddb\" (UniqueName: \"kubernetes.io/projected/a3c931cb-95f1-4dc4-8730-204905465bcf-kube-api-access-j2ddb\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.495857 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.495833 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbp86" Jul 02 23:23:09.926464 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.926288 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbp86"] Jul 02 23:23:09.930915 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:09.930892 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod070538f1_749e_45d0_bf8d_f3219633dd6d.slice/crio-f78e1a2b98957facd37dbad81002a9430d8038e44ae4585b3234369468d07ce2 WatchSource:0}: Error finding container f78e1a2b98957facd37dbad81002a9430d8038e44ae4585b3234369468d07ce2: Status 404 returned error can't find the container with id f78e1a2b98957facd37dbad81002a9430d8038e44ae4585b3234369468d07ce2 Jul 02 23:23:09.955870 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:09.955852 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:09.955970 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:09.955959 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:09.956016 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:09.956007 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls podName:a3c931cb-95f1-4dc4-8730-204905465bcf nodeName:}" failed. No retries permitted until 2026-07-02 23:23:10.95599319 +0000 UTC m=+37.695285625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls") pod "dns-default-cgggg" (UID: "a3c931cb-95f1-4dc4-8730-204905465bcf") : secret "dns-default-metrics-tls" not found Jul 02 23:23:10.003893 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.003855 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbp86" event={"ID":"070538f1-749e-45d0-bf8d-f3219633dd6d","Type":"ContainerStarted","Data":"f78e1a2b98957facd37dbad81002a9430d8038e44ae4585b3234369468d07ce2"} Jul 02 23:23:10.006755 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.006725 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerStarted","Data":"343c3c57a13668741fa26aa809cd3180751a68a89e5d819e5d9f594651fecced"} Jul 02 23:23:10.676756 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.676724 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:10.676899 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.676761 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:10.676899 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.676760 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:10.679380 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.679340 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:23:10.680135 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.680095 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:23:10.680135 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.680119 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:23:10.680321 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.680141 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:23:10.680321 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.680170 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z4l4l\"" Jul 02 23:23:10.680321 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.680120 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p57lg\"" Jul 02 23:23:10.963422 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:10.963338 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:10.963795 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:10.963499 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:10.963795 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:10.963585 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls podName:a3c931cb-95f1-4dc4-8730-204905465bcf nodeName:}" failed. No retries permitted until 2026-07-02 23:23:12.963565118 +0000 UTC m=+39.702857571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls") pod "dns-default-cgggg" (UID: "a3c931cb-95f1-4dc4-8730-204905465bcf") : secret "dns-default-metrics-tls" not found Jul 02 23:23:11.013332 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:11.013288 2555 generic.go:356] "Generic (PLEG): container finished" podID="507bb56b-96aa-4ab3-b284-b589f3959f4e" containerID="343c3c57a13668741fa26aa809cd3180751a68a89e5d819e5d9f594651fecced" exitCode=0 Jul 02 23:23:11.013500 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:11.013388 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerDied","Data":"343c3c57a13668741fa26aa809cd3180751a68a89e5d819e5d9f594651fecced"} Jul 02 23:23:12.019269 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.019232 2555 generic.go:356] "Generic (PLEG): container finished" podID="507bb56b-96aa-4ab3-b284-b589f3959f4e" containerID="64a7ba6017c019c1398d48f804457585450fe390daf1cc35b6881110a993dcc0" exitCode=0 Jul 02 23:23:12.019727 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.019310 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerDied","Data":"64a7ba6017c019c1398d48f804457585450fe390daf1cc35b6881110a993dcc0"} Jul 02 23:23:12.479428 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.479356 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b7245"] Jul 02 23:23:12.482820 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.482799 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.485092 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.485073 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 23:23:12.485229 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.485218 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 23:23:12.485281 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.485268 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-242f6\"" Jul 02 23:23:12.485329 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.485315 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 23:23:12.485676 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.485658 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 23:23:12.489518 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.489500 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b7245"] Jul 02 23:23:12.673981 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.673953 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fm4x\" (UniqueName: \"kubernetes.io/projected/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-api-access-5fm4x\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.674078 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.674044 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a296e165-7e89-4db7-8e0b-c86cea87bf46-data-volume\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.674113 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.674085 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.674145 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.674120 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.674174 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.674163 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a296e165-7e89-4db7-8e0b-c86cea87bf46-crio-socket\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.775288 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.775253 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a296e165-7e89-4db7-8e0b-c86cea87bf46-data-volume\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.775431 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.775297 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.775431 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.775318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.775431 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.775348 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a296e165-7e89-4db7-8e0b-c86cea87bf46-crio-socket\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.775431 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.775386 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fm4x\" (UniqueName: \"kubernetes.io/projected/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-api-access-5fm4x\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.775431 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:12.775396 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:23:12.775646 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:12.775464 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:13.275446601 +0000 UTC m=+40.014739061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : configmap "kube-rbac-proxy" not found Jul 02 23:23:12.775646 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:12.775494 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:12.775646 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:12.775550 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:13.275532151 +0000 UTC m=+40.014824596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:12.775646 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.775614 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a296e165-7e89-4db7-8e0b-c86cea87bf46-data-volume\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.775646 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.775636 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a296e165-7e89-4db7-8e0b-c86cea87bf46-crio-socket\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.784180 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.784157 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fm4x\" (UniqueName: \"kubernetes.io/projected/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-api-access-5fm4x\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:12.976776 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:12.976729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:12.976875 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:12.976859 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:12.976913 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:12.976909 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls podName:a3c931cb-95f1-4dc4-8730-204905465bcf nodeName:}" failed. No retries permitted until 2026-07-02 23:23:16.976897277 +0000 UTC m=+43.716189712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls") pod "dns-default-cgggg" (UID: "a3c931cb-95f1-4dc4-8730-204905465bcf") : secret "dns-default-metrics-tls" not found Jul 02 23:23:13.026046 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.026017 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbp86" event={"ID":"070538f1-749e-45d0-bf8d-f3219633dd6d","Type":"ContainerStarted","Data":"a4501b555940ad8e293ee4b3cf3d2c0d6cd64d89de64e7c4dd302feacb4349e6"} Jul 02 23:23:13.029117 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.029093 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" event={"ID":"507bb56b-96aa-4ab3-b284-b589f3959f4e","Type":"ContainerStarted","Data":"0c00e2158cc2b887f364eb6adf78c8eaca6372de08b90f2acc0410484788f2c1"} Jul 02 23:23:13.062981 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.062939 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbp86" podStartSLOduration=1.542519371 podStartE2EDuration="4.062927223s" totalImagesPullingTime="2.520407852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:13.038981979 +0000 UTC m=+39.778274436" watchObservedRunningTime="2026-07-02 23:23:13.062927223 +0000 UTC m=+39.802219680" Jul 02 23:23:13.063250 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.063229 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qrhzq" podStartSLOduration=9.159943592 podStartE2EDuration="39.063223923s" totalImagesPullingTime="29.903280331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:13.061718927 +0000 UTC m=+39.801011384" watchObservedRunningTime="2026-07-02 23:23:13.063223923 +0000 UTC m=+39.802516380" Jul 02 23:23:13.279239 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.279180 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:13.279239 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.279217 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:13.279355 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:13.279282 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:23:13.279355 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:13.279309 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:13.279355 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:13.279333 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:14.279319599 +0000 UTC m=+41.018612036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : configmap "kube-rbac-proxy" not found Jul 02 23:23:13.279355 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:13.279346 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:14.279340308 +0000 UTC m=+41.018632743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:13.341432 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.341412 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf"] Jul 02 23:23:13.361936 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.361913 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf"] Jul 02 23:23:13.362032 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.362008 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" Jul 02 23:23:13.364407 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.364389 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 02 23:23:13.364407 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.364401 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 02 23:23:13.364563 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.364405 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-k9lbv\"" Jul 02 23:23:13.379513 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.379495 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h82z5\" (UniqueName: \"kubernetes.io/projected/6aaac3b1-ed0e-44b7-be54-0aecd1f3b527-kube-api-access-h82z5\") pod \"migrator-d57d58cc9-57bsf\" (UID: \"6aaac3b1-ed0e-44b7-be54-0aecd1f3b527\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" Jul 02 23:23:13.480091 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.480072 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h82z5\" (UniqueName: \"kubernetes.io/projected/6aaac3b1-ed0e-44b7-be54-0aecd1f3b527-kube-api-access-h82z5\") pod \"migrator-d57d58cc9-57bsf\" (UID: \"6aaac3b1-ed0e-44b7-be54-0aecd1f3b527\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" Jul 02 23:23:13.487132 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.487115 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h82z5\" (UniqueName: \"kubernetes.io/projected/6aaac3b1-ed0e-44b7-be54-0aecd1f3b527-kube-api-access-h82z5\") pod \"migrator-d57d58cc9-57bsf\" (UID: \"6aaac3b1-ed0e-44b7-be54-0aecd1f3b527\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" Jul 02 23:23:13.670598 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.670579 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" Jul 02 23:23:13.797412 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:13.797388 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf"] Jul 02 23:23:13.799885 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:13.799859 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aaac3b1_ed0e_44b7_be54_0aecd1f3b527.slice/crio-051d8c3145ba6640cd37ec73fe5c0569f3db2a0c4a161efffc5683bd06f2793a WatchSource:0}: Error finding container 051d8c3145ba6640cd37ec73fe5c0569f3db2a0c4a161efffc5683bd06f2793a: Status 404 returned error can't find the container with id 051d8c3145ba6640cd37ec73fe5c0569f3db2a0c4a161efffc5683bd06f2793a Jul 02 23:23:14.032293 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:14.032222 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" event={"ID":"6aaac3b1-ed0e-44b7-be54-0aecd1f3b527","Type":"ContainerStarted","Data":"051d8c3145ba6640cd37ec73fe5c0569f3db2a0c4a161efffc5683bd06f2793a"} Jul 02 23:23:14.284943 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:14.284887 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:14.284943 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:14.284931 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:14.285093 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:14.284979 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:23:14.285093 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:14.285017 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:14.285093 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:14.285029 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:16.285014272 +0000 UTC m=+43.024306707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : configmap "kube-rbac-proxy" not found Jul 02 23:23:14.285093 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:14.285072 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:16.285057085 +0000 UTC m=+43.024349521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:14.770791 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:14.770770 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qpvc2_2dfdc72a-9586-4e0b-b68d-7008640b7770/dns-node-resolver/0.log" Jul 02 23:23:15.370131 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.370103 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pnb5p_401fe81a-d58d-430b-bb94-f73fdce48999/node-ca/0.log" Jul 02 23:23:15.915966 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.915915 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-dpcf5"] Jul 02 23:23:15.938322 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.938305 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-dpcf5"] Jul 02 23:23:15.938420 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.938394 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:15.940908 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.940893 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 02 23:23:15.941871 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.941852 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 02 23:23:15.941979 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.941872 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 02 23:23:15.941979 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.941908 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 02 23:23:15.942079 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.942002 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tvf7r\"" Jul 02 23:23:15.998974 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.998955 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8581f98-aeda-4e89-a77c-42dd36b656a4-signing-key\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:15.999048 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.998987 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8581f98-aeda-4e89-a77c-42dd36b656a4-signing-cabundle\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:15.999048 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:15.999012 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2fhv\" (UniqueName: \"kubernetes.io/projected/c8581f98-aeda-4e89-a77c-42dd36b656a4-kube-api-access-x2fhv\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:16.038507 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.038484 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" event={"ID":"6aaac3b1-ed0e-44b7-be54-0aecd1f3b527","Type":"ContainerStarted","Data":"c1e95f3e0e14ce77d38ce735e392b8c730ceb95cf50299a7d50b1815040e06ad"} Jul 02 23:23:16.038582 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.038523 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" event={"ID":"6aaac3b1-ed0e-44b7-be54-0aecd1f3b527","Type":"ContainerStarted","Data":"f33775b0759237aa339bffd8b66b55db88eeb31d7c2b26cc774cf88a2d8bd6eb"} Jul 02 23:23:16.054806 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.054770 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-57bsf" podStartSLOduration=1.449266898 podStartE2EDuration="3.054757609s" totalImagesPullingTime="1.605490711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:16.054393582 +0000 UTC m=+42.793686056" watchObservedRunningTime="2026-07-02 23:23:16.054757609 +0000 UTC m=+42.794050066" Jul 02 23:23:16.100095 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.100080 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8581f98-aeda-4e89-a77c-42dd36b656a4-signing-key\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:16.100187 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.100118 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8581f98-aeda-4e89-a77c-42dd36b656a4-signing-cabundle\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:16.100187 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.100136 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2fhv\" (UniqueName: \"kubernetes.io/projected/c8581f98-aeda-4e89-a77c-42dd36b656a4-kube-api-access-x2fhv\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:16.100733 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.100713 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8581f98-aeda-4e89-a77c-42dd36b656a4-signing-cabundle\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:16.102480 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.102452 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8581f98-aeda-4e89-a77c-42dd36b656a4-signing-key\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:16.111279 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.111258 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2fhv\" (UniqueName: \"kubernetes.io/projected/c8581f98-aeda-4e89-a77c-42dd36b656a4-kube-api-access-x2fhv\") pod \"service-ca-758f599c5-dpcf5\" (UID: \"c8581f98-aeda-4e89-a77c-42dd36b656a4\") " pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:16.246992 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.246948 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-dpcf5" Jul 02 23:23:16.301823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.301790 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:16.301823 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.301826 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:16.301955 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:16.301911 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:23:16.301955 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:16.301919 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:16.302015 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:16.301965 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:20.301952653 +0000 UTC m=+47.041245088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : configmap "kube-rbac-proxy" not found Jul 02 23:23:16.302015 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:16.301983 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:20.301973655 +0000 UTC m=+47.041266090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:16.371707 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.371680 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbp86_070538f1-749e-45d0-bf8d-f3219633dd6d/serve-healthcheck-canary/0.log" Jul 02 23:23:16.395542 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:16.395518 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-dpcf5"] Jul 02 23:23:16.398318 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:16.398295 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8581f98_aeda_4e89_a77c_42dd36b656a4.slice/crio-92e1d1885c2310e251744bf4082d4a76b3cee3800530ee6f0033a632b5d2b923 WatchSource:0}: Error finding container 92e1d1885c2310e251744bf4082d4a76b3cee3800530ee6f0033a632b5d2b923: Status 404 returned error can't find the container with id 92e1d1885c2310e251744bf4082d4a76b3cee3800530ee6f0033a632b5d2b923 Jul 02 23:23:17.007516 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:17.007491 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:17.007660 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:17.007635 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:17.007705 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:17.007694 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls podName:a3c931cb-95f1-4dc4-8730-204905465bcf nodeName:}" failed. No retries permitted until 2026-07-02 23:23:25.007676193 +0000 UTC m=+51.746968630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls") pod "dns-default-cgggg" (UID: "a3c931cb-95f1-4dc4-8730-204905465bcf") : secret "dns-default-metrics-tls" not found Jul 02 23:23:17.041630 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:17.041602 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-dpcf5" event={"ID":"c8581f98-aeda-4e89-a77c-42dd36b656a4","Type":"ContainerStarted","Data":"92e1d1885c2310e251744bf4082d4a76b3cee3800530ee6f0033a632b5d2b923"} Jul 02 23:23:19.048006 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:19.047971 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-dpcf5" event={"ID":"c8581f98-aeda-4e89-a77c-42dd36b656a4","Type":"ContainerStarted","Data":"6b632ff7be524c1923f24f3bf917f090b953a54bd2b7d903c51204a0ca1eb368"} Jul 02 23:23:19.063929 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:19.063883 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-dpcf5" podStartSLOduration=1.546077044 podStartE2EDuration="4.063863958s" totalImagesPullingTime="2.517786914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:19.062530105 +0000 UTC m=+45.801822563" watchObservedRunningTime="2026-07-02 23:23:19.063863958 +0000 UTC m=+45.803156416" Jul 02 23:23:20.332408 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:20.332353 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:20.333008 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:20.332427 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:20.333008 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:20.332575 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:20.333008 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:20.332632 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls podName:a296e165-7e89-4db7-8e0b-c86cea87bf46 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:28.332613503 +0000 UTC m=+55.071905951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b7245" (UID: "a296e165-7e89-4db7-8e0b-c86cea87bf46") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:20.333457 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:20.333420 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a296e165-7e89-4db7-8e0b-c86cea87bf46-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:25.064078 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:25.064045 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:25.066304 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:25.066286 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c931cb-95f1-4dc4-8730-204905465bcf-metrics-tls\") pod \"dns-default-cgggg\" (UID: \"a3c931cb-95f1-4dc4-8730-204905465bcf\") " pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:25.089839 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:25.089813 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:25.239846 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:25.239779 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cgggg"] Jul 02 23:23:25.242468 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:25.242445 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c931cb_95f1_4dc4_8730_204905465bcf.slice/crio-b5927275f485bce0c5ce4a7500573dd38481f6d7b0eac1bd816c55e78dab30b8 WatchSource:0}: Error finding container b5927275f485bce0c5ce4a7500573dd38481f6d7b0eac1bd816c55e78dab30b8: Status 404 returned error can't find the container with id b5927275f485bce0c5ce4a7500573dd38481f6d7b0eac1bd816c55e78dab30b8 Jul 02 23:23:26.064897 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:26.064863 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cgggg" event={"ID":"a3c931cb-95f1-4dc4-8730-204905465bcf","Type":"ContainerStarted","Data":"b5927275f485bce0c5ce4a7500573dd38481f6d7b0eac1bd816c55e78dab30b8"} Jul 02 23:23:28.076147 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:28.076116 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cgggg" event={"ID":"a3c931cb-95f1-4dc4-8730-204905465bcf","Type":"ContainerStarted","Data":"a492016810237e8ffd9f1304a143613dd44576188d1453061a59eb8ce804462d"} Jul 02 23:23:28.076573 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:28.076184 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cgggg" event={"ID":"a3c931cb-95f1-4dc4-8730-204905465bcf","Type":"ContainerStarted","Data":"d21e2c6bdbe9bc1c8bbffe3089c5a1139ac9f7c1004bd024d693b6f6d5870b27"} Jul 02 23:23:28.076573 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:28.076258 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:28.091946 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:28.091904 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cgggg" podStartSLOduration=16.885333508 podStartE2EDuration="19.091891905s" totalImagesPullingTime="2.206558397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:28.091717927 +0000 UTC m=+54.831010384" watchObservedRunningTime="2026-07-02 23:23:28.091891905 +0000 UTC m=+54.831184361" Jul 02 23:23:28.387741 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:28.387686 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:28.389902 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:28.389885 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a296e165-7e89-4db7-8e0b-c86cea87bf46-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b7245\" (UID: \"a296e165-7e89-4db7-8e0b-c86cea87bf46\") " pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:28.410134 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:28.410115 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b7245" Jul 02 23:23:28.527323 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:28.527287 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b7245"] Jul 02 23:23:28.530017 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:28.529989 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda296e165_7e89_4db7_8e0b_c86cea87bf46.slice/crio-653933cea9227929a4d956eca1bbc2280d51879f55a4a9c732463c584dcabd5b WatchSource:0}: Error finding container 653933cea9227929a4d956eca1bbc2280d51879f55a4a9c732463c584dcabd5b: Status 404 returned error can't find the container with id 653933cea9227929a4d956eca1bbc2280d51879f55a4a9c732463c584dcabd5b Jul 02 23:23:29.081862 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:29.081775 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b7245" event={"ID":"a296e165-7e89-4db7-8e0b-c86cea87bf46","Type":"ContainerStarted","Data":"653933cea9227929a4d956eca1bbc2280d51879f55a4a9c732463c584dcabd5b"} Jul 02 23:23:33.097321 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:33.097291 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b7245" event={"ID":"a296e165-7e89-4db7-8e0b-c86cea87bf46","Type":"ContainerStarted","Data":"f927721883ac25607210ecd50dc15d0cf9ad4697b8f7452bfc720cc8364f9895"} Jul 02 23:23:33.097656 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:33.097350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b7245" event={"ID":"a296e165-7e89-4db7-8e0b-c86cea87bf46","Type":"ContainerStarted","Data":"94bf2843daa12180475bd1611963dc7aa37f210ee4256bc90ecd8792d92a237c"} Jul 02 23:23:34.521379 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.521333 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs"] Jul 02 23:23:34.539981 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.539946 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs"] Jul 02 23:23:34.540148 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.540051 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" Jul 02 23:23:34.543684 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.543656 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 02 23:23:34.543684 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.543676 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 23:23:34.543919 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.543689 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tfnsx\"" Jul 02 23:23:34.543919 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.543658 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 23:23:34.543919 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.543682 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 23:23:34.637324 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.637288 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e4fd519-ff61-412d-a52f-d3fe32b4dc11-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dd9d9847d-2txqs\" (UID: \"2e4fd519-ff61-412d-a52f-d3fe32b4dc11\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" Jul 02 23:23:34.637515 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.637357 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tmth\" (UniqueName: \"kubernetes.io/projected/2e4fd519-ff61-412d-a52f-d3fe32b4dc11-kube-api-access-8tmth\") pod \"managed-serviceaccount-addon-agent-6dd9d9847d-2txqs\" (UID: \"2e4fd519-ff61-412d-a52f-d3fe32b4dc11\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" Jul 02 23:23:34.638298 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.638277 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59c99698c5-rnqdk"] Jul 02 23:23:34.641038 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.641020 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.643406 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.643384 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 23:23:34.643511 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.643425 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 23:23:34.643511 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.643487 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j45fm\"" Jul 02 23:23:34.643676 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.643660 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 23:23:34.649788 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.649766 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 23:23:34.655201 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.655182 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59c99698c5-rnqdk"] Jul 02 23:23:34.738825 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.738786 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d03b71c8-8d20-482f-a075-21ab7ae4976d-trusted-ca\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.739011 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.738833 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tpdw\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-kube-api-access-9tpdw\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.739011 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.738973 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e4fd519-ff61-412d-a52f-d3fe32b4dc11-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dd9d9847d-2txqs\" (UID: \"2e4fd519-ff61-412d-a52f-d3fe32b4dc11\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" Jul 02 23:23:34.739147 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.739070 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-registry-tls\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.739147 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.739103 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d03b71c8-8d20-482f-a075-21ab7ae4976d-ca-trust-extracted\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.739147 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.739135 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d03b71c8-8d20-482f-a075-21ab7ae4976d-image-registry-private-configuration\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.739318 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.739171 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d03b71c8-8d20-482f-a075-21ab7ae4976d-registry-certificates\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.739318 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.739192 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d03b71c8-8d20-482f-a075-21ab7ae4976d-installation-pull-secrets\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.739318 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.739275 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tmth\" (UniqueName: \"kubernetes.io/projected/2e4fd519-ff61-412d-a52f-d3fe32b4dc11-kube-api-access-8tmth\") pod \"managed-serviceaccount-addon-agent-6dd9d9847d-2txqs\" (UID: \"2e4fd519-ff61-412d-a52f-d3fe32b4dc11\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" Jul 02 23:23:34.739499 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.739314 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-bound-sa-token\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.741794 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.741768 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e4fd519-ff61-412d-a52f-d3fe32b4dc11-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dd9d9847d-2txqs\" (UID: \"2e4fd519-ff61-412d-a52f-d3fe32b4dc11\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" Jul 02 23:23:34.749768 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.749742 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tmth\" (UniqueName: \"kubernetes.io/projected/2e4fd519-ff61-412d-a52f-d3fe32b4dc11-kube-api-access-8tmth\") pod \"managed-serviceaccount-addon-agent-6dd9d9847d-2txqs\" (UID: \"2e4fd519-ff61-412d-a52f-d3fe32b4dc11\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" Jul 02 23:23:34.840405 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.840374 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-bound-sa-token\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.840569 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.840445 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d03b71c8-8d20-482f-a075-21ab7ae4976d-trusted-ca\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.840569 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.840474 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpdw\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-kube-api-access-9tpdw\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.840709 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.840681 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-registry-tls\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.840764 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.840747 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d03b71c8-8d20-482f-a075-21ab7ae4976d-ca-trust-extracted\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.840879 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.840857 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d03b71c8-8d20-482f-a075-21ab7ae4976d-image-registry-private-configuration\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.840963 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.840893 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d03b71c8-8d20-482f-a075-21ab7ae4976d-registry-certificates\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.840963 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.840918 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d03b71c8-8d20-482f-a075-21ab7ae4976d-installation-pull-secrets\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.841064 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.841031 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d03b71c8-8d20-482f-a075-21ab7ae4976d-ca-trust-extracted\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.841572 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.841490 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d03b71c8-8d20-482f-a075-21ab7ae4976d-trusted-ca\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.841716 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.841689 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d03b71c8-8d20-482f-a075-21ab7ae4976d-registry-certificates\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.843803 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.843782 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d03b71c8-8d20-482f-a075-21ab7ae4976d-image-registry-private-configuration\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.843901 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.843781 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d03b71c8-8d20-482f-a075-21ab7ae4976d-installation-pull-secrets\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.843901 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.843839 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-registry-tls\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.852733 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.852713 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tpdw\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-kube-api-access-9tpdw\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.855408 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.855388 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d03b71c8-8d20-482f-a075-21ab7ae4976d-bound-sa-token\") pod \"image-registry-59c99698c5-rnqdk\" (UID: \"d03b71c8-8d20-482f-a075-21ab7ae4976d\") " pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:34.858265 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.858245 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" Jul 02 23:23:34.954395 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:34.954349 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:35.001433 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:35.001409 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8mhj9" Jul 02 23:23:35.506515 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:35.506492 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59c99698c5-rnqdk"] Jul 02 23:23:35.509213 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:35.509188 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd03b71c8_8d20_482f_a075_21ab7ae4976d.slice/crio-1b279bee4cacc8ead12cf5c856e9090373646b7cc585aab3ffd49deefc01e270 WatchSource:0}: Error finding container 1b279bee4cacc8ead12cf5c856e9090373646b7cc585aab3ffd49deefc01e270: Status 404 returned error can't find the container with id 1b279bee4cacc8ead12cf5c856e9090373646b7cc585aab3ffd49deefc01e270 Jul 02 23:23:35.518202 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:35.518180 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs"] Jul 02 23:23:35.520768 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:35.520747 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e4fd519_ff61_412d_a52f_d3fe32b4dc11.slice/crio-6d744689ca9da36b646be9f69ee021d24739077f85dc43c5c9832ebb87bf1795 WatchSource:0}: Error finding container 6d744689ca9da36b646be9f69ee021d24739077f85dc43c5c9832ebb87bf1795: Status 404 returned error can't find the container with id 6d744689ca9da36b646be9f69ee021d24739077f85dc43c5c9832ebb87bf1795 Jul 02 23:23:36.107802 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:36.107763 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" event={"ID":"2e4fd519-ff61-412d-a52f-d3fe32b4dc11","Type":"ContainerStarted","Data":"6d744689ca9da36b646be9f69ee021d24739077f85dc43c5c9832ebb87bf1795"} Jul 02 23:23:36.109745 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:36.109724 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b7245" event={"ID":"a296e165-7e89-4db7-8e0b-c86cea87bf46","Type":"ContainerStarted","Data":"cad5848106e09f5f7ac6e4daf53bd3336ef43fc41a5a2e5edb4f376a7aa3d425"} Jul 02 23:23:36.111186 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:36.111168 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" event={"ID":"d03b71c8-8d20-482f-a075-21ab7ae4976d","Type":"ContainerStarted","Data":"d3bf7a01c4ffbdfe183461fbf7c282218e3724e08e5fe47951dcd50a4fb7e87f"} Jul 02 23:23:36.111268 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:36.111198 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" event={"ID":"d03b71c8-8d20-482f-a075-21ab7ae4976d","Type":"ContainerStarted","Data":"1b279bee4cacc8ead12cf5c856e9090373646b7cc585aab3ffd49deefc01e270"} Jul 02 23:23:36.111268 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:36.111226 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:23:36.128579 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:36.128529 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-b7245" podStartSLOduration=17.347077153 podStartE2EDuration="24.128516049s" totalImagesPullingTime="6.781438896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:23:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:36.127920538 +0000 UTC m=+62.867213002" watchObservedRunningTime="2026-07-02 23:23:36.128516049 +0000 UTC m=+62.867808505" Jul 02 23:23:36.144089 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:36.144053 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" podStartSLOduration=2.144042359 podStartE2EDuration="2.144042359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:36.143665879 +0000 UTC m=+62.882958358" watchObservedRunningTime="2026-07-02 23:23:36.144042359 +0000 UTC m=+62.883334815" Jul 02 23:23:38.084468 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.084433 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cgggg" Jul 02 23:23:38.471846 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.471753 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:38.472005 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.471846 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:38.474182 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.474161 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:23:38.474301 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.474163 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:23:38.484128 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.484109 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4cf293e-362f-45b5-aad9-e3a34a8f52f8-metrics-certs\") pod \"network-metrics-daemon-88h28\" (UID: \"f4cf293e-362f-45b5-aad9-e3a34a8f52f8\") " pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:38.484253 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.484234 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1b6e71e-c320-46f2-b9f7-07fd0a76cabc-original-pull-secret\") pod \"global-pull-secret-syncer-hhcrv\" (UID: \"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc\") " pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:38.572894 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.572868 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:38.575077 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.575060 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:23:38.585637 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.585619 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:23:38.587449 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.587433 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhcrv" Jul 02 23:23:38.596926 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.596782 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z4l4l\"" Jul 02 23:23:38.596979 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.596962 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6s9d\" (UniqueName: \"kubernetes.io/projected/11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb-kube-api-access-d6s9d\") pod \"network-check-target-6qsmv\" (UID: \"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb\") " pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:38.602572 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.602555 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p57lg\"" Jul 02 23:23:38.605299 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.605282 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88h28" Jul 02 23:23:38.611128 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.611110 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:38.736805 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:38.736773 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b6e71e_c320_46f2_b9f7_07fd0a76cabc.slice/crio-755117b2a0accc2e1b2930b78ebb13eca4131ae59714d1cef9ace748a4a76e8e WatchSource:0}: Error finding container 755117b2a0accc2e1b2930b78ebb13eca4131ae59714d1cef9ace748a4a76e8e: Status 404 returned error can't find the container with id 755117b2a0accc2e1b2930b78ebb13eca4131ae59714d1cef9ace748a4a76e8e Jul 02 23:23:38.738188 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.738157 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hhcrv"] Jul 02 23:23:38.752917 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.752885 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-88h28"] Jul 02 23:23:38.757104 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:38.757077 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cf293e_362f_45b5_aad9_e3a34a8f52f8.slice/crio-d6d9356f6055adec7b5e385d1f36537ab234f1eedd203662a29d58ce35b8fb38 WatchSource:0}: Error finding container d6d9356f6055adec7b5e385d1f36537ab234f1eedd203662a29d58ce35b8fb38: Status 404 returned error can't find the container with id d6d9356f6055adec7b5e385d1f36537ab234f1eedd203662a29d58ce35b8fb38 Jul 02 23:23:38.765741 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:38.765722 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6qsmv"] Jul 02 23:23:38.768314 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:38.768293 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11dde8ca_04a8_4bdb_9ed9_48fb1d53f3eb.slice/crio-aa000381433a9371a618aa95b44928efa4b9b5b3e31a91106d399b75a9d589b9 WatchSource:0}: Error finding container aa000381433a9371a618aa95b44928efa4b9b5b3e31a91106d399b75a9d589b9: Status 404 returned error can't find the container with id aa000381433a9371a618aa95b44928efa4b9b5b3e31a91106d399b75a9d589b9 Jul 02 23:23:39.123166 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:39.123124 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" event={"ID":"2e4fd519-ff61-412d-a52f-d3fe32b4dc11","Type":"ContainerStarted","Data":"4a98582418ae4625078fbc7caa6d6f81ebfa835f937707460be838d080ae6a6b"} Jul 02 23:23:39.124399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:39.124337 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6qsmv" event={"ID":"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb","Type":"ContainerStarted","Data":"aa000381433a9371a618aa95b44928efa4b9b5b3e31a91106d399b75a9d589b9"} Jul 02 23:23:39.125904 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:39.125868 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88h28" event={"ID":"f4cf293e-362f-45b5-aad9-e3a34a8f52f8","Type":"ContainerStarted","Data":"d6d9356f6055adec7b5e385d1f36537ab234f1eedd203662a29d58ce35b8fb38"} Jul 02 23:23:39.127356 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:39.127315 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hhcrv" event={"ID":"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc","Type":"ContainerStarted","Data":"755117b2a0accc2e1b2930b78ebb13eca4131ae59714d1cef9ace748a4a76e8e"} Jul 02 23:23:39.137071 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:39.137030 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dd9d9847d-2txqs" podStartSLOduration=2.495150572 podStartE2EDuration="5.137017046s" totalImagesPullingTime="2.641866474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:39.136239564 +0000 UTC m=+65.875532020" watchObservedRunningTime="2026-07-02 23:23:39.137017046 +0000 UTC m=+65.876309514" Jul 02 23:23:40.133578 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.133537 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88h28" event={"ID":"f4cf293e-362f-45b5-aad9-e3a34a8f52f8","Type":"ContainerStarted","Data":"cd4866756440440e4c6607e950379eeef50908c3dd5354e36eed62b2f6228024"} Jul 02 23:23:40.133951 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.133605 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88h28" event={"ID":"f4cf293e-362f-45b5-aad9-e3a34a8f52f8","Type":"ContainerStarted","Data":"1edc274f91195255826526a8747d644ed68932187b00b2f9421506f31a1bca51"} Jul 02 23:23:40.152899 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.150033 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-88h28" podStartSLOduration=65.168988442 podStartE2EDuration="1m6.150012049s" totalImagesPullingTime="981.023607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:40.148002623 +0000 UTC m=+66.887295080" watchObservedRunningTime="2026-07-02 23:23:40.150012049 +0000 UTC m=+66.889304507" Jul 02 23:23:40.264857 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.264821 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp"] Jul 02 23:23:40.284005 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.283965 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp"] Jul 02 23:23:40.284132 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.284102 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" Jul 02 23:23:40.287077 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.286804 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 23:23:40.287077 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.286876 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cgzfh\"" Jul 02 23:23:40.392528 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.392439 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2f28e20a-a3e6-4989-9b5f-a87f1b3669c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ptpbp\" (UID: \"2f28e20a-a3e6-4989-9b5f-a87f1b3669c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" Jul 02 23:23:40.493798 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.493757 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2f28e20a-a3e6-4989-9b5f-a87f1b3669c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ptpbp\" (UID: \"2f28e20a-a3e6-4989-9b5f-a87f1b3669c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" Jul 02 23:23:40.493957 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:40.493923 2555 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 02 23:23:40.494024 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:40.494005 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f28e20a-a3e6-4989-9b5f-a87f1b3669c9-tls-certificates podName:2f28e20a-a3e6-4989-9b5f-a87f1b3669c9 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:40.993988881 +0000 UTC m=+67.733281322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/2f28e20a-a3e6-4989-9b5f-a87f1b3669c9-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-ptpbp" (UID: "2f28e20a-a3e6-4989-9b5f-a87f1b3669c9") : secret "prometheus-operator-admission-webhook-tls" not found Jul 02 23:23:40.997631 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:40.997593 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2f28e20a-a3e6-4989-9b5f-a87f1b3669c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ptpbp\" (UID: \"2f28e20a-a3e6-4989-9b5f-a87f1b3669c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" Jul 02 23:23:41.000511 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:41.000488 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2f28e20a-a3e6-4989-9b5f-a87f1b3669c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-ptpbp\" (UID: \"2f28e20a-a3e6-4989-9b5f-a87f1b3669c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" Jul 02 23:23:41.194744 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:41.194711 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" Jul 02 23:23:43.702975 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:43.702942 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f28e20a_a3e6_4989_9b5f_a87f1b3669c9.slice/crio-ce0e0f7ffa54eae33a802513151dae9f7a3f7a8fd4f95ba46c7faf5b9c70bb0d WatchSource:0}: Error finding container ce0e0f7ffa54eae33a802513151dae9f7a3f7a8fd4f95ba46c7faf5b9c70bb0d: Status 404 returned error can't find the container with id ce0e0f7ffa54eae33a802513151dae9f7a3f7a8fd4f95ba46c7faf5b9c70bb0d Jul 02 23:23:43.703629 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:43.703607 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp"] Jul 02 23:23:44.150044 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:44.150006 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6qsmv" event={"ID":"11dde8ca-04a8-4bdb-9ed9-48fb1d53f3eb","Type":"ContainerStarted","Data":"57e5e3a4a3528a86bc1e3e38d8f05136da4d9fb3a7540109791be555ea7b09cb"} Jul 02 23:23:44.150195 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:44.150123 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:23:44.151456 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:44.151421 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hhcrv" event={"ID":"c1b6e71e-c320-46f2-b9f7-07fd0a76cabc","Type":"ContainerStarted","Data":"25825a0bde057559c2b777e2c966502c5bdc3ea24cffedea8a4f15aad8ab1ab8"} Jul 02 23:23:44.152444 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:44.152426 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" event={"ID":"2f28e20a-a3e6-4989-9b5f-a87f1b3669c9","Type":"ContainerStarted","Data":"ce0e0f7ffa54eae33a802513151dae9f7a3f7a8fd4f95ba46c7faf5b9c70bb0d"} Jul 02 23:23:44.164055 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:44.164021 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6qsmv" podStartSLOduration=65.359320649 podStartE2EDuration="1m10.164008784s" totalImagesPullingTime="4.804688135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:44.163561247 +0000 UTC m=+70.902853704" watchObservedRunningTime="2026-07-02 23:23:44.164008784 +0000 UTC m=+70.903301239" Jul 02 23:23:44.176271 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:44.176231 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hhcrv" podStartSLOduration=65.343657652 podStartE2EDuration="1m10.176220858s" totalImagesPullingTime="4.832563206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:44.175675075 +0000 UTC m=+70.914967534" watchObservedRunningTime="2026-07-02 23:23:44.176220858 +0000 UTC m=+70.915513315" Jul 02 23:23:45.157276 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.157239 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" event={"ID":"2f28e20a-a3e6-4989-9b5f-a87f1b3669c9","Type":"ContainerStarted","Data":"7d543fa2adb282b97b757aff185329b7f53964944bab2784fddc1e1d8357a3d9"} Jul 02 23:23:45.176416 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.176352 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" podStartSLOduration=4.135363402 podStartE2EDuration="5.176337549s" totalImagesPullingTime="1.040974147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:45.175592303 +0000 UTC m=+71.914884782" watchObservedRunningTime="2026-07-02 23:23:45.176337549 +0000 UTC m=+71.915630005" Jul 02 23:23:45.179953 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.179923 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-x469x"] Jul 02 23:23:45.182402 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.182384 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-x469x" Jul 02 23:23:45.184552 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.184530 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-szb6f\"" Jul 02 23:23:45.184650 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.184584 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 23:23:45.184650 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.184602 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 23:23:45.191814 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.191795 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-x469x"] Jul 02 23:23:45.229139 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.229110 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kmd8\" (UniqueName: \"kubernetes.io/projected/feec3cd6-90b3-4633-b539-3a4a8c827706-kube-api-access-4kmd8\") pod \"downloads-685bfb5f64-x469x\" (UID: \"feec3cd6-90b3-4633-b539-3a4a8c827706\") " pod="openshift-console/downloads-685bfb5f64-x469x" Jul 02 23:23:45.330412 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.330384 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kmd8\" (UniqueName: \"kubernetes.io/projected/feec3cd6-90b3-4633-b539-3a4a8c827706-kube-api-access-4kmd8\") pod \"downloads-685bfb5f64-x469x\" (UID: \"feec3cd6-90b3-4633-b539-3a4a8c827706\") " pod="openshift-console/downloads-685bfb5f64-x469x" Jul 02 23:23:45.338128 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.338104 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kmd8\" (UniqueName: \"kubernetes.io/projected/feec3cd6-90b3-4633-b539-3a4a8c827706-kube-api-access-4kmd8\") pod \"downloads-685bfb5f64-x469x\" (UID: \"feec3cd6-90b3-4633-b539-3a4a8c827706\") " pod="openshift-console/downloads-685bfb5f64-x469x" Jul 02 23:23:45.491417 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.491335 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-x469x" Jul 02 23:23:45.611654 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:45.611631 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-x469x"] Jul 02 23:23:45.613986 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:45.613961 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeec3cd6_90b3_4633_b539_3a4a8c827706.slice/crio-b18ce166fbf061e956ad8b21a0851a62a04c325639d7ec2871fc4a7c99b36564 WatchSource:0}: Error finding container b18ce166fbf061e956ad8b21a0851a62a04c325639d7ec2871fc4a7c99b36564: Status 404 returned error can't find the container with id b18ce166fbf061e956ad8b21a0851a62a04c325639d7ec2871fc4a7c99b36564 Jul 02 23:23:46.161961 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:46.161931 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-x469x" event={"ID":"feec3cd6-90b3-4633-b539-3a4a8c827706","Type":"ContainerStarted","Data":"b18ce166fbf061e956ad8b21a0851a62a04c325639d7ec2871fc4a7c99b36564"} Jul 02 23:23:46.162303 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:46.162107 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" Jul 02 23:23:46.166474 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:46.166452 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-ptpbp" Jul 02 23:23:50.653414 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.653351 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4"] Jul 02 23:23:50.658856 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.658833 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.662338 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.662313 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 02 23:23:50.662502 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.662321 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7d28c\"" Jul 02 23:23:50.663205 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.663174 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 02 23:23:50.663609 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.663591 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 02 23:23:50.664093 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.663878 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 23:23:50.664093 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.664049 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 23:23:50.664259 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.664174 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 23:23:50.669054 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.669036 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4"] Jul 02 23:23:50.672688 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.672669 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2f8gc"] Jul 02 23:23:50.676451 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.676434 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.678776 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.678755 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 23:23:50.678876 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.678764 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 23:23:50.679032 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.679015 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 23:23:50.679103 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.679090 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sgptq\"" Jul 02 23:23:50.775645 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775617 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4ck8\" (UniqueName: \"kubernetes.io/projected/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-api-access-b4ck8\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.775813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775657 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-tls\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.775813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775685 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-wtmp\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.775813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bfb829dd-f5e9-4066-84c1-edf3440398cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.775813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775806 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.775990 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775839 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfb829dd-f5e9-4066-84c1-edf3440398cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.775990 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775897 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcz82\" (UniqueName: \"kubernetes.io/projected/b018168b-5e45-4312-bf72-06c8e9fdbb9d-kube-api-access-zcz82\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.775990 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775949 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.775990 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.775976 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.776163 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.776004 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b018168b-5e45-4312-bf72-06c8e9fdbb9d-metrics-client-ca\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.776163 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.776080 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-root\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.776163 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.776112 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-sys\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.776163 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.776146 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.776380 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.776198 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-textfile\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.776380 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.776228 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877191 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-root\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877236 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-sys\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877269 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877307 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-textfile\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877337 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877386 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4ck8\" (UniqueName: \"kubernetes.io/projected/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-api-access-b4ck8\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877411 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-tls\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877433 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-wtmp\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877463 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bfb829dd-f5e9-4066-84c1-edf3440398cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877494 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877520 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfb829dd-f5e9-4066-84c1-edf3440398cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.878141 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.877559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz82\" (UniqueName: \"kubernetes.io/projected/b018168b-5e45-4312-bf72-06c8e9fdbb9d-kube-api-access-zcz82\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.878843 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.878325 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-root\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.878843 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.878415 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-sys\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.885479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.879134 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.885479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.879455 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-textfile\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.885479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.880073 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bfb829dd-f5e9-4066-84c1-edf3440398cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.885479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.881946 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfb829dd-f5e9-4066-84c1-edf3440398cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.885479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.882993 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.885479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.883227 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-tls\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.885479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.885406 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.885479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.885459 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.885953 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.885489 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b018168b-5e45-4312-bf72-06c8e9fdbb9d-metrics-client-ca\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.885953 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.885657 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-wtmp\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.886688 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.886265 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.886688 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.886608 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b018168b-5e45-4312-bf72-06c8e9fdbb9d-metrics-client-ca\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.889149 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.889124 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4ck8\" (UniqueName: \"kubernetes.io/projected/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-api-access-b4ck8\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.889710 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.889668 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b018168b-5e45-4312-bf72-06c8e9fdbb9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.910413 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.908675 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcz82\" (UniqueName: \"kubernetes.io/projected/b018168b-5e45-4312-bf72-06c8e9fdbb9d-kube-api-access-zcz82\") pod \"node-exporter-2f8gc\" (UID: \"b018168b-5e45-4312-bf72-06c8e9fdbb9d\") " pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:50.910413 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.908817 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfb829dd-f5e9-4066-84c1-edf3440398cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-s6zw4\" (UID: \"bfb829dd-f5e9-4066-84c1-edf3440398cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.971723 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.971693 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" Jul 02 23:23:50.987415 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:50.987338 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2f8gc" Jul 02 23:23:51.005021 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:51.004911 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb018168b_5e45_4312_bf72_06c8e9fdbb9d.slice/crio-e37fbbca0575b263181e1dfc7fd63559aef3661c1575f8110a3171ba4a692781 WatchSource:0}: Error finding container e37fbbca0575b263181e1dfc7fd63559aef3661c1575f8110a3171ba4a692781: Status 404 returned error can't find the container with id e37fbbca0575b263181e1dfc7fd63559aef3661c1575f8110a3171ba4a692781 Jul 02 23:23:51.124578 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.124550 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4"] Jul 02 23:23:51.126597 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:51.126562 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb829dd_f5e9_4066_84c1_edf3440398cc.slice/crio-32f30c42a037c5552b34bb2b955483f8d069299167c524d3c3c5d636d83c562a WatchSource:0}: Error finding container 32f30c42a037c5552b34bb2b955483f8d069299167c524d3c3c5d636d83c562a: Status 404 returned error can't find the container with id 32f30c42a037c5552b34bb2b955483f8d069299167c524d3c3c5d636d83c562a Jul 02 23:23:51.183604 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.183526 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2f8gc" event={"ID":"b018168b-5e45-4312-bf72-06c8e9fdbb9d","Type":"ContainerStarted","Data":"e37fbbca0575b263181e1dfc7fd63559aef3661c1575f8110a3171ba4a692781"} Jul 02 23:23:51.184744 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.184721 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" event={"ID":"bfb829dd-f5e9-4066-84c1-edf3440398cc","Type":"ContainerStarted","Data":"32f30c42a037c5552b34bb2b955483f8d069299167c524d3c3c5d636d83c562a"} Jul 02 23:23:51.734272 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.734242 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:23:51.743673 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.743644 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.746063 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746038 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-szlp8\"" Jul 02 23:23:51.746209 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746095 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 23:23:51.746308 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746291 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 23:23:51.746546 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746291 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 23:23:51.746903 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746881 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 23:23:51.746996 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746931 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 23:23:51.746996 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746885 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 23:23:51.747123 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746884 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 23:23:51.747123 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.746887 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 23:23:51.747353 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.747331 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 23:23:51.754975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.754957 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:23:51.892595 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892560 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-out\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892741 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892602 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892741 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892741 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892688 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892741 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892741 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892737 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892994 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892765 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892994 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892796 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892994 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892826 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892994 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892861 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892994 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892908 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-web-config\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892994 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892930 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69b82\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-kube-api-access-69b82\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.892994 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.892958 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993721 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993647 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993721 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993700 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-web-config\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69b82\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-kube-api-access-69b82\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993756 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993790 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-out\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993847 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993883 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.993944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993935 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.994305 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993963 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.994305 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.993993 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.994305 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.994026 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.994472 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.994452 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.995648 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.995340 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:51.997059 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:23:51.997035 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle podName:5ed16a19-f3d1-43d8-8564-a22d8d4ad13a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:52.497010663 +0000 UTC m=+79.236303115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a") : configmap references non-existent config key: ca-bundle.crt Jul 02 23:23:51.998632 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:51.998589 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.000999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.000981 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.003105 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.003077 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.003219 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.003108 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-web-config\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.003400 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.003351 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.004031 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.003629 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.004742 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.004319 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.004742 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.004512 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.006387 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.006327 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-out\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.012714 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.012669 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69b82\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-kube-api-access-69b82\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.499119 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.499082 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.500026 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.499997 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.658900 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.658869 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:23:52.808864 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:52.808843 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:23:52.809434 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:52.809400 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed16a19_f3d1_43d8_8564_a22d8d4ad13a.slice/crio-d23166854623a29e4d1ec8c32e27b5a1736c625f7388a06a8cc19c3011dac60f WatchSource:0}: Error finding container d23166854623a29e4d1ec8c32e27b5a1736c625f7388a06a8cc19c3011dac60f: Status 404 returned error can't find the container with id d23166854623a29e4d1ec8c32e27b5a1736c625f7388a06a8cc19c3011dac60f Jul 02 23:23:53.197239 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.197143 2555 generic.go:356] "Generic (PLEG): container finished" podID="b018168b-5e45-4312-bf72-06c8e9fdbb9d" containerID="125bf74f2de73b0e091eef616cc123cda57b2a1789fdaa26add90088b8a3aaeb" exitCode=0 Jul 02 23:23:53.197416 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.197246 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2f8gc" event={"ID":"b018168b-5e45-4312-bf72-06c8e9fdbb9d","Type":"ContainerDied","Data":"125bf74f2de73b0e091eef616cc123cda57b2a1789fdaa26add90088b8a3aaeb"} Jul 02 23:23:53.200707 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.200579 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" event={"ID":"bfb829dd-f5e9-4066-84c1-edf3440398cc","Type":"ContainerStarted","Data":"2c999fe2c7385813a771f7bb96fd26a293565347b7cf799d03ff6bb9a46950e4"} Jul 02 23:23:53.200707 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.200624 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" event={"ID":"bfb829dd-f5e9-4066-84c1-edf3440398cc","Type":"ContainerStarted","Data":"cf83995bd9425a1fca9825641f34b295f979c2b84765c96281659b8f33c3a831"} Jul 02 23:23:53.200707 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.200653 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" event={"ID":"bfb829dd-f5e9-4066-84c1-edf3440398cc","Type":"ContainerStarted","Data":"1bec304004ff743a746891271d4e9ff383a68781ab36779c5ca6b821008abf7b"} Jul 02 23:23:53.202330 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.202303 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerStarted","Data":"d23166854623a29e4d1ec8c32e27b5a1736c625f7388a06a8cc19c3011dac60f"} Jul 02 23:23:53.234056 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.233326 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-s6zw4" podStartSLOduration=1.650568686 podStartE2EDuration="3.233306507s" totalImagesPullingTime="1.582737821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:53.232705507 +0000 UTC m=+79.971997991" watchObservedRunningTime="2026-07-02 23:23:53.233306507 +0000 UTC m=+79.972598966" Jul 02 23:23:53.636287 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.633302 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66b64698c7-m6k4d"] Jul 02 23:23:53.638570 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.638550 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.641281 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.641260 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 02 23:23:53.641454 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.641260 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fti6edmuh02ug\"" Jul 02 23:23:53.641700 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.641674 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 02 23:23:53.641805 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.641761 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 02 23:23:53.641987 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.641971 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-sss5b\"" Jul 02 23:23:53.642173 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.642157 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 02 23:23:53.642514 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.642497 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 02 23:23:53.651667 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.649636 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66b64698c7-m6k4d"] Jul 02 23:23:53.710152 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.710089 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.710152 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.710144 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.710345 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.710186 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-tls\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.710345 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.710222 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5061fb7d-1dfa-4671-aadd-523c08eb4822-metrics-client-ca\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.710345 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.710249 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-grpc-tls\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.710345 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.710280 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.710345 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.710338 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.710635 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.710392 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgf6j\" (UniqueName: \"kubernetes.io/projected/5061fb7d-1dfa-4671-aadd-523c08eb4822-kube-api-access-dgf6j\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.810933 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.810898 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.810933 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.810938 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgf6j\" (UniqueName: \"kubernetes.io/projected/5061fb7d-1dfa-4671-aadd-523c08eb4822-kube-api-access-dgf6j\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.811559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.810965 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.811559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.810999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.811559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.811017 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-tls\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.811559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.811051 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5061fb7d-1dfa-4671-aadd-523c08eb4822-metrics-client-ca\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.811559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.811081 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-grpc-tls\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.811559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.811120 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.811883 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.811840 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5061fb7d-1dfa-4671-aadd-523c08eb4822-metrics-client-ca\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.814236 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.814210 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.814607 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.814584 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-grpc-tls\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.814668 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.814649 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.815762 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.815741 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.817093 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.817051 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-tls\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.817226 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.817201 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5061fb7d-1dfa-4671-aadd-523c08eb4822-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.818486 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.818445 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgf6j\" (UniqueName: \"kubernetes.io/projected/5061fb7d-1dfa-4671-aadd-523c08eb4822-kube-api-access-dgf6j\") pod \"thanos-querier-66b64698c7-m6k4d\" (UID: \"5061fb7d-1dfa-4671-aadd-523c08eb4822\") " pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:53.956316 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:53.956239 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:23:54.200283 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.200244 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66b64698c7-m6k4d"] Jul 02 23:23:54.201929 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:54.201901 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5061fb7d_1dfa_4671_aadd_523c08eb4822.slice/crio-bdd2fb3d102623ebc68e47a0612283d93610b61403b0459cf7393807ca6d9c9a WatchSource:0}: Error finding container bdd2fb3d102623ebc68e47a0612283d93610b61403b0459cf7393807ca6d9c9a: Status 404 returned error can't find the container with id bdd2fb3d102623ebc68e47a0612283d93610b61403b0459cf7393807ca6d9c9a Jul 02 23:23:54.209957 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.209924 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerStarted","Data":"df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939"} Jul 02 23:23:54.212340 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.212315 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2f8gc" event={"ID":"b018168b-5e45-4312-bf72-06c8e9fdbb9d","Type":"ContainerStarted","Data":"e6f6b933825c3a6cad87aeca4fddf4af97502edb6d7c856bd2ba78ba327cdbd4"} Jul 02 23:23:54.212458 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.212381 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2f8gc" event={"ID":"b018168b-5e45-4312-bf72-06c8e9fdbb9d","Type":"ContainerStarted","Data":"f0b361a2151fdc0cfa5882b1516d2038f98f22db15203b40f2846bd1814f97d5"} Jul 02 23:23:54.213841 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.213816 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" event={"ID":"5061fb7d-1dfa-4671-aadd-523c08eb4822","Type":"ContainerStarted","Data":"bdd2fb3d102623ebc68e47a0612283d93610b61403b0459cf7393807ca6d9c9a"} Jul 02 23:23:54.500074 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.499907 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2f8gc" podStartSLOduration=3.3056191520000002 podStartE2EDuration="4.49988504s" totalImagesPullingTime="1.114128805s" totalInitContainerRuntime="80.137083ms" isStatefulPod=true podCreationTimestamp="2026-07-02 23:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:54.247010629 +0000 UTC m=+80.986303087" watchObservedRunningTime="2026-07-02 23:23:54.49988504 +0000 UTC m=+81.239177496" Jul 02 23:23:54.501058 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.501037 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69b5965ff7-9wrdw"] Jul 02 23:23:54.504652 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.504633 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.507056 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.507035 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 23:23:54.507188 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.507097 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 23:23:54.507319 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.507297 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 23:23:54.507397 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.507343 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 23:23:54.507551 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.507538 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fqzxt\"" Jul 02 23:23:54.507809 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.507789 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 23:23:54.511380 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.511346 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69b5965ff7-9wrdw"] Jul 02 23:23:54.617662 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.617626 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t56sh\" (UniqueName: \"kubernetes.io/projected/9a453dd1-7520-48ed-a232-b43bdd78f854-kube-api-access-t56sh\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.617836 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.617684 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-serving-cert\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.617836 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.617729 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-oauth-config\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.617836 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.617807 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-service-ca\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.617976 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.617858 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-console-config\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.617976 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.617887 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-oauth-serving-cert\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.718773 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.718732 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-console-config\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.718955 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.718786 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-oauth-serving-cert\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.718955 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.718833 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t56sh\" (UniqueName: \"kubernetes.io/projected/9a453dd1-7520-48ed-a232-b43bdd78f854-kube-api-access-t56sh\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.718955 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.718869 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-serving-cert\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.718955 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.718898 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-oauth-config\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.718955 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.718941 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-service-ca\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.719794 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.719626 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-console-config\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.719794 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.719647 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-oauth-serving-cert\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.719794 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.719664 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-service-ca\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.721805 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.721785 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-oauth-config\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.721895 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.721849 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-serving-cert\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.726541 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.726519 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t56sh\" (UniqueName: \"kubernetes.io/projected/9a453dd1-7520-48ed-a232-b43bdd78f854-kube-api-access-t56sh\") pod \"console-69b5965ff7-9wrdw\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.817121 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.817065 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:23:54.970828 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.969411 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m"] Jul 02 23:23:54.974416 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.974390 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:54.975335 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.975307 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69b5965ff7-9wrdw"] Jul 02 23:23:54.977565 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.977540 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 23:23:54.977663 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.977549 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 02 23:23:54.977729 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.977549 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 02 23:23:54.979150 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.979124 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 02 23:23:54.979372 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.979331 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4vensm8pca7og\"" Jul 02 23:23:54.979481 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.979418 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9j8lb\"" Jul 02 23:23:54.982122 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:54.982087 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a453dd1_7520_48ed_a232_b43bdd78f854.slice/crio-7f612c20cdf3ef988725f01ad2021ee42ce4ee9b460e44dbdd3c02fc8bf7a9cf WatchSource:0}: Error finding container 7f612c20cdf3ef988725f01ad2021ee42ce4ee9b460e44dbdd3c02fc8bf7a9cf: Status 404 returned error can't find the container with id 7f612c20cdf3ef988725f01ad2021ee42ce4ee9b460e44dbdd3c02fc8bf7a9cf Jul 02 23:23:54.990560 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:54.990538 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m"] Jul 02 23:23:55.121720 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.121645 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-audit-log\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.121720 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.121699 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.121906 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.121732 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnq4z\" (UniqueName: \"kubernetes.io/projected/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-kube-api-access-fnq4z\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.121906 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.121764 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-client-ca-bundle\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.121906 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.121790 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-secret-metrics-server-tls\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.122009 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.121964 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-secret-metrics-server-client-certs\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.122009 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.121989 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-metrics-server-audit-profiles\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.220822 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.220782 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerID="df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939" exitCode=0 Jul 02 23:23:55.221021 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.220879 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerDied","Data":"df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939"} Jul 02 23:23:55.222469 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.222444 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-secret-metrics-server-client-certs\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.222549 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.222484 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-metrics-server-audit-profiles\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.222549 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.222515 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-audit-log\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.222666 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.222547 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.222666 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.222574 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnq4z\" (UniqueName: \"kubernetes.io/projected/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-kube-api-access-fnq4z\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.222666 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.222598 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-client-ca-bundle\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.222666 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.222618 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-secret-metrics-server-tls\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.224153 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.224123 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69b5965ff7-9wrdw" event={"ID":"9a453dd1-7520-48ed-a232-b43bdd78f854","Type":"ContainerStarted","Data":"7f612c20cdf3ef988725f01ad2021ee42ce4ee9b460e44dbdd3c02fc8bf7a9cf"} Jul 02 23:23:55.224258 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.224163 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-metrics-server-audit-profiles\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.225397 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.224768 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.225397 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.224970 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-audit-log\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.226058 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.226036 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-secret-metrics-server-client-certs\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.227142 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.227086 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-client-ca-bundle\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.227142 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.227134 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-secret-metrics-server-tls\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.232298 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.232274 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnq4z\" (UniqueName: \"kubernetes.io/projected/ba025c2d-86d1-4d5d-b6db-246e60d7ec24-kube-api-access-fnq4z\") pod \"metrics-server-58ffd5b7cd-9vl2m\" (UID: \"ba025c2d-86d1-4d5d-b6db-246e60d7ec24\") " pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.288192 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.287969 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:23:55.448862 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:55.448833 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m"] Jul 02 23:23:55.451830 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:23:55.451790 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba025c2d_86d1_4d5d_b6db_246e60d7ec24.slice/crio-c20dc2a1c8290ed63d6f8db44893248b8c9940a4b33411f765ddf94676e01c50 WatchSource:0}: Error finding container c20dc2a1c8290ed63d6f8db44893248b8c9940a4b33411f765ddf94676e01c50: Status 404 returned error can't find the container with id c20dc2a1c8290ed63d6f8db44893248b8c9940a4b33411f765ddf94676e01c50 Jul 02 23:23:56.233333 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:56.233296 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" event={"ID":"ba025c2d-86d1-4d5d-b6db-246e60d7ec24","Type":"ContainerStarted","Data":"c20dc2a1c8290ed63d6f8db44893248b8c9940a4b33411f765ddf94676e01c50"} Jul 02 23:23:57.121559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:23:57.120736 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59c99698c5-rnqdk" Jul 02 23:24:06.277845 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:06.277793 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" event={"ID":"5061fb7d-1dfa-4671-aadd-523c08eb4822","Type":"ContainerStarted","Data":"f4702077f2c325471eab80b72cbdf1f849fd59babed7e2730a649782aa219eca"} Jul 02 23:24:06.281159 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:06.280028 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-x469x" event={"ID":"feec3cd6-90b3-4633-b539-3a4a8c827706","Type":"ContainerStarted","Data":"33e8d50e149eb4084075ac1911f9c240d6785b118704bea09a2f740926a16e30"} Jul 02 23:24:06.281159 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:06.281096 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-x469x" Jul 02 23:24:06.284037 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:06.284013 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerStarted","Data":"907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db"} Jul 02 23:24:06.294030 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:06.293952 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-x469x" Jul 02 23:24:06.298280 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:06.296673 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-x469x" podStartSLOduration=1.232888897 podStartE2EDuration="21.296653064s" totalImagesPullingTime="20.063764167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:24:06.294608949 +0000 UTC m=+93.033901397" watchObservedRunningTime="2026-07-02 23:24:06.296653064 +0000 UTC m=+93.035945521" Jul 02 23:24:07.290976 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.290935 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" event={"ID":"ba025c2d-86d1-4d5d-b6db-246e60d7ec24","Type":"ContainerStarted","Data":"0c9b073f5da4e9e8953b5eb95476cb58c5adbdb98c4a26750beeaa2aae3a17bb"} Jul 02 23:24:07.295050 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.295017 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerStarted","Data":"78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126"} Jul 02 23:24:07.295200 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.295090 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerStarted","Data":"36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc"} Jul 02 23:24:07.295200 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.295124 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerStarted","Data":"44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a"} Jul 02 23:24:07.295200 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.295162 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerStarted","Data":"8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af"} Jul 02 23:24:07.296915 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.296864 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69b5965ff7-9wrdw" event={"ID":"9a453dd1-7520-48ed-a232-b43bdd78f854","Type":"ContainerStarted","Data":"6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa"} Jul 02 23:24:07.299654 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.299522 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" event={"ID":"5061fb7d-1dfa-4671-aadd-523c08eb4822","Type":"ContainerStarted","Data":"df82c29a7abaa4376bfc22d6b151bca565754506e303a536d28500d8e686332f"} Jul 02 23:24:07.299654 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.299592 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" event={"ID":"5061fb7d-1dfa-4671-aadd-523c08eb4822","Type":"ContainerStarted","Data":"39fff7c1c51fe56d5deb3f20417a64919d3f4362556967dc8255353e4c775b4a"} Jul 02 23:24:07.308415 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.308351 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" podStartSLOduration=1.84505513 podStartE2EDuration="13.308334385s" totalImagesPullingTime="11.463279255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:24:07.306396876 +0000 UTC m=+94.045689333" watchObservedRunningTime="2026-07-02 23:24:07.308334385 +0000 UTC m=+94.047626843" Jul 02 23:24:07.322629 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:07.322583 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69b5965ff7-9wrdw" podStartSLOduration=1.786158589 podStartE2EDuration="13.322565936s" totalImagesPullingTime="11.536407347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:24:07.321384015 +0000 UTC m=+94.060676468" watchObservedRunningTime="2026-07-02 23:24:07.322565936 +0000 UTC m=+94.061858395" Jul 02 23:24:09.313567 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:09.313522 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" event={"ID":"5061fb7d-1dfa-4671-aadd-523c08eb4822","Type":"ContainerStarted","Data":"55a6d1b5b2e2493a79a85e12bfdf47a5860fb167895a63c4e022679e02595bb3"} Jul 02 23:24:09.314083 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:09.313673 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:24:09.314083 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:09.313722 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" event={"ID":"5061fb7d-1dfa-4671-aadd-523c08eb4822","Type":"ContainerStarted","Data":"8c4e124cfa310270656fd8229c318179c947b470333e542a111e0beb6638e36c"} Jul 02 23:24:09.314083 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:09.313759 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" event={"ID":"5061fb7d-1dfa-4671-aadd-523c08eb4822","Type":"ContainerStarted","Data":"bb39491f51d0947259494c55b8fa55e41f03864bf4a6bdd6b76600a00de459b6"} Jul 02 23:24:09.317955 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:09.317931 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerStarted","Data":"3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a"} Jul 02 23:24:09.335989 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:09.335946 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" podStartSLOduration=3.196816807 podStartE2EDuration="16.335929944s" totalImagesPullingTime="13.139113137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:24:09.333794791 +0000 UTC m=+96.073087248" watchObservedRunningTime="2026-07-02 23:24:09.335929944 +0000 UTC m=+96.075222399" Jul 02 23:24:09.360920 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:09.360873 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.196118015 podStartE2EDuration="18.360852168s" totalImagesPullingTime="13.138696547s" totalInitContainerRuntime="26.037606ms" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:24:09.359237101 +0000 UTC m=+96.098529590" watchObservedRunningTime="2026-07-02 23:24:09.360852168 +0000 UTC m=+96.100144625" Jul 02 23:24:14.817669 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:14.817630 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:24:14.818101 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:14.817695 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:24:14.822347 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:14.822323 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:24:15.159643 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:15.159577 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6qsmv" Jul 02 23:24:15.288697 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:15.288672 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:24:15.288853 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:15.288717 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:24:15.330322 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:15.330298 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66b64698c7-m6k4d" Jul 02 23:24:15.348315 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:15.348295 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:24:16.046218 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:16.046182 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69b5965ff7-9wrdw"] Jul 02 23:24:35.293158 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:35.293127 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:24:35.297274 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:35.297255 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-58ffd5b7cd-9vl2m" Jul 02 23:24:42.372475 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.372415 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-69b5965ff7-9wrdw" podUID="9a453dd1-7520-48ed-a232-b43bdd78f854" containerName="console" containerID="cri-o://6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa" gracePeriod=15 Jul 02 23:24:42.655490 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.655434 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:24:42.721843 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.721819 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-service-ca\") pod \"9a453dd1-7520-48ed-a232-b43bdd78f854\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " Jul 02 23:24:42.721959 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.721851 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-console-config\") pod \"9a453dd1-7520-48ed-a232-b43bdd78f854\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " Jul 02 23:24:42.721959 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.721880 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-serving-cert\") pod \"9a453dd1-7520-48ed-a232-b43bdd78f854\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " Jul 02 23:24:42.721959 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.721916 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-oauth-config\") pod \"9a453dd1-7520-48ed-a232-b43bdd78f854\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " Jul 02 23:24:42.721959 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.721937 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-oauth-serving-cert\") pod \"9a453dd1-7520-48ed-a232-b43bdd78f854\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " Jul 02 23:24:42.722135 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.721966 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t56sh\" (UniqueName: \"kubernetes.io/projected/9a453dd1-7520-48ed-a232-b43bdd78f854-kube-api-access-t56sh\") pod \"9a453dd1-7520-48ed-a232-b43bdd78f854\" (UID: \"9a453dd1-7520-48ed-a232-b43bdd78f854\") " Jul 02 23:24:42.722344 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.722298 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-console-config" (OuterVolumeSpecName: "console-config") pod "9a453dd1-7520-48ed-a232-b43bdd78f854" (UID: "9a453dd1-7520-48ed-a232-b43bdd78f854"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:24:42.722344 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.722336 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a453dd1-7520-48ed-a232-b43bdd78f854" (UID: "9a453dd1-7520-48ed-a232-b43bdd78f854"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:24:42.722510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.722387 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a453dd1-7520-48ed-a232-b43bdd78f854" (UID: "9a453dd1-7520-48ed-a232-b43bdd78f854"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:24:42.724192 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.724170 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a453dd1-7520-48ed-a232-b43bdd78f854" (UID: "9a453dd1-7520-48ed-a232-b43bdd78f854"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:24:42.724266 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.724235 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a453dd1-7520-48ed-a232-b43bdd78f854-kube-api-access-t56sh" (OuterVolumeSpecName: "kube-api-access-t56sh") pod "9a453dd1-7520-48ed-a232-b43bdd78f854" (UID: "9a453dd1-7520-48ed-a232-b43bdd78f854"). InnerVolumeSpecName "kube-api-access-t56sh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:24:42.724266 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.724249 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a453dd1-7520-48ed-a232-b43bdd78f854" (UID: "9a453dd1-7520-48ed-a232-b43bdd78f854"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:24:42.822816 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.822796 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-service-ca\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:24:42.822929 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.822909 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-console-config\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:24:42.822999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.822936 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-serving-cert\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:24:42.822999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.822954 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a453dd1-7520-48ed-a232-b43bdd78f854-console-oauth-config\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:24:42.822999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.822969 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a453dd1-7520-48ed-a232-b43bdd78f854-oauth-serving-cert\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:24:42.822999 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:42.822984 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t56sh\" (UniqueName: \"kubernetes.io/projected/9a453dd1-7520-48ed-a232-b43bdd78f854-kube-api-access-t56sh\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:24:43.456481 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.456448 2555 generic.go:356] "Generic (PLEG): container finished" podID="9a453dd1-7520-48ed-a232-b43bdd78f854" containerID="6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa" exitCode=0 Jul 02 23:24:43.456882 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.456496 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69b5965ff7-9wrdw" Jul 02 23:24:43.456882 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.456547 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69b5965ff7-9wrdw" event={"ID":"9a453dd1-7520-48ed-a232-b43bdd78f854","Type":"ContainerDied","Data":"6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa"} Jul 02 23:24:43.456882 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.456600 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69b5965ff7-9wrdw" event={"ID":"9a453dd1-7520-48ed-a232-b43bdd78f854","Type":"ContainerDied","Data":"7f612c20cdf3ef988725f01ad2021ee42ce4ee9b460e44dbdd3c02fc8bf7a9cf"} Jul 02 23:24:43.456882 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.456621 2555 scope.go:117] "RemoveContainer" containerID="6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa" Jul 02 23:24:43.465274 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.465258 2555 scope.go:117] "RemoveContainer" containerID="6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa" Jul 02 23:24:43.465584 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:24:43.465563 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa\": container with ID starting with 6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa not found: ID does not exist" containerID="6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa" Jul 02 23:24:43.465645 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.465594 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa"} err="failed to get container status \"6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa\": rpc error: code = NotFound desc = could not find container \"6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa\": container with ID starting with 6da6fa4dbc6053e2e5823060635068bc53277583d8d6c0341876b08e22e894fa not found: ID does not exist" Jul 02 23:24:43.475301 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.475278 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69b5965ff7-9wrdw"] Jul 02 23:24:43.478906 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.478886 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69b5965ff7-9wrdw"] Jul 02 23:24:43.682712 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:24:43.682682 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a453dd1-7520-48ed-a232-b43bdd78f854" path="/var/lib/kubelet/pods/9a453dd1-7520-48ed-a232-b43bdd78f854/volumes" Jul 02 23:25:10.865308 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:10.865273 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:25:10.865837 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:10.865595 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="alertmanager" containerID="cri-o://907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db" gracePeriod=120 Jul 02 23:25:10.865837 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:10.865675 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy-metric" containerID="cri-o://78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126" gracePeriod=120 Jul 02 23:25:10.865837 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:10.865733 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy" containerID="cri-o://36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc" gracePeriod=120 Jul 02 23:25:10.865837 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:10.865721 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy-web" containerID="cri-o://44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a" gracePeriod=120 Jul 02 23:25:10.865837 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:10.865688 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="prom-label-proxy" containerID="cri-o://3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a" gracePeriod=120 Jul 02 23:25:10.865837 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:10.865750 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="config-reloader" containerID="cri-o://8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af" gracePeriod=120 Jul 02 23:25:11.575221 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575187 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerID="3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a" exitCode=0 Jul 02 23:25:11.575221 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575212 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerID="78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126" exitCode=0 Jul 02 23:25:11.575221 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575219 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerID="36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc" exitCode=0 Jul 02 23:25:11.575221 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575227 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerID="8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af" exitCode=0 Jul 02 23:25:11.575510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575235 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerID="907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db" exitCode=0 Jul 02 23:25:11.575510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575256 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerDied","Data":"3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a"} Jul 02 23:25:11.575510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575324 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerDied","Data":"78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126"} Jul 02 23:25:11.575510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575352 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerDied","Data":"36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc"} Jul 02 23:25:11.575510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575393 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerDied","Data":"8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af"} Jul 02 23:25:11.575510 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:11.575417 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerDied","Data":"907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db"} Jul 02 23:25:12.104276 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.104252 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.153807 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.153781 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.153975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.153811 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-cluster-tls-config\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.153975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.153837 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-metrics-client-ca\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.153975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.153864 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154155 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.153992 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154155 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154042 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-web\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154155 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154070 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-tls-assets\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154155 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154101 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-web-config\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154155 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154133 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69b82\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-kube-api-access-69b82\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154585 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154168 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-volume\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154585 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154196 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-main-db\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154585 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154221 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-main-tls\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154585 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154221 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:12.154585 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154285 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-out\") pod \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\" (UID: \"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a\") " Jul 02 23:25:12.154585 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154564 2555 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-metrics-client-ca\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.154886 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.154643 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:12.156386 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.156248 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:25:12.157541 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.157489 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-out" (OuterVolumeSpecName: "config-out") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:25:12.157738 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.157696 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:12.157842 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.157811 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:12.158022 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.157983 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-volume" (OuterVolumeSpecName: "config-volume") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:12.158130 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.158100 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:12.158448 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.158420 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:12.158712 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.158686 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-kube-api-access-69b82" (OuterVolumeSpecName: "kube-api-access-69b82") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "kube-api-access-69b82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:25:12.159993 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.159952 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:25:12.162330 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.162221 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:12.169713 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.169663 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-web-config" (OuterVolumeSpecName: "web-config") pod "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" (UID: "5ed16a19-f3d1-43d8-8564-a22d8d4ad13a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:12.255600 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255572 2555 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-out\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255600 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255596 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255600 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255606 2555 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-cluster-tls-config\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255615 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255626 2555 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255634 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255643 2555 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-tls-assets\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255651 2555 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-web-config\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255658 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69b82\" (UniqueName: \"kubernetes.io/projected/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-kube-api-access-69b82\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255666 2555 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-config-volume\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255674 2555 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-alertmanager-main-db\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.255801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.255682 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a-secret-alertmanager-main-tls\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:25:12.582437 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.582409 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerID="44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a" exitCode=0 Jul 02 23:25:12.582600 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.582443 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerDied","Data":"44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a"} Jul 02 23:25:12.582600 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.582481 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.582600 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.582500 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ed16a19-f3d1-43d8-8564-a22d8d4ad13a","Type":"ContainerDied","Data":"d23166854623a29e4d1ec8c32e27b5a1736c625f7388a06a8cc19c3011dac60f"} Jul 02 23:25:12.582600 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.582532 2555 scope.go:117] "RemoveContainer" containerID="3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a" Jul 02 23:25:12.592927 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.592911 2555 scope.go:117] "RemoveContainer" containerID="78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126" Jul 02 23:25:12.599875 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.599857 2555 scope.go:117] "RemoveContainer" containerID="36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc" Jul 02 23:25:12.605219 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.605197 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:25:12.607190 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.607175 2555 scope.go:117] "RemoveContainer" containerID="44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a" Jul 02 23:25:12.611460 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.611440 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:25:12.613950 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.613933 2555 scope.go:117] "RemoveContainer" containerID="8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af" Jul 02 23:25:12.620845 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.620831 2555 scope.go:117] "RemoveContainer" containerID="907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db" Jul 02 23:25:12.627653 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.627639 2555 scope.go:117] "RemoveContainer" containerID="df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939" Jul 02 23:25:12.634057 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.634043 2555 scope.go:117] "RemoveContainer" containerID="3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a" Jul 02 23:25:12.634297 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:25:12.634273 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a\": container with ID starting with 3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a not found: ID does not exist" containerID="3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a" Jul 02 23:25:12.634399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.634303 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a"} err="failed to get container status \"3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a\": rpc error: code = NotFound desc = could not find container \"3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a\": container with ID starting with 3292a7ad6c44aacb5d9e3a264dfe9b768c43ea620da1a2e64fc2f5f81521218a not found: ID does not exist" Jul 02 23:25:12.634399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.634336 2555 scope.go:117] "RemoveContainer" containerID="78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126" Jul 02 23:25:12.634603 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:25:12.634587 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126\": container with ID starting with 78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126 not found: ID does not exist" containerID="78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126" Jul 02 23:25:12.634645 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.634607 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126"} err="failed to get container status \"78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126\": rpc error: code = NotFound desc = could not find container \"78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126\": container with ID starting with 78c1308f3ef0833a232b1ae94e50aa9bd17add6d7144831c9ec6e192454ea126 not found: ID does not exist" Jul 02 23:25:12.634645 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.634621 2555 scope.go:117] "RemoveContainer" containerID="36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc" Jul 02 23:25:12.634863 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:25:12.634846 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc\": container with ID starting with 36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc not found: ID does not exist" containerID="36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc" Jul 02 23:25:12.634901 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.634868 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc"} err="failed to get container status \"36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc\": rpc error: code = NotFound desc = could not find container \"36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc\": container with ID starting with 36c8a3203f5dc72b6c03b982529f210b721ecca8e45b1d20dd1dec6cbbc18bfc not found: ID does not exist" Jul 02 23:25:12.634901 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.634883 2555 scope.go:117] "RemoveContainer" containerID="44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a" Jul 02 23:25:12.635115 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:25:12.635101 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a\": container with ID starting with 44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a not found: ID does not exist" containerID="44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a" Jul 02 23:25:12.635156 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.635117 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a"} err="failed to get container status \"44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a\": rpc error: code = NotFound desc = could not find container \"44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a\": container with ID starting with 44a525d072a1826b2437aad120fc99d3254375972dc9c48cb93ecad8d8415c2a not found: ID does not exist" Jul 02 23:25:12.635156 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.635130 2555 scope.go:117] "RemoveContainer" containerID="8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af" Jul 02 23:25:12.635409 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:25:12.635391 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af\": container with ID starting with 8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af not found: ID does not exist" containerID="8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af" Jul 02 23:25:12.635478 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.635412 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af"} err="failed to get container status \"8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af\": rpc error: code = NotFound desc = could not find container \"8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af\": container with ID starting with 8c96dcfc2922eb21ac55922f97329c974f8ea92acca49cd5f9ff97958282a2af not found: ID does not exist" Jul 02 23:25:12.635478 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.635424 2555 scope.go:117] "RemoveContainer" containerID="907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db" Jul 02 23:25:12.635745 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:25:12.635725 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db\": container with ID starting with 907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db not found: ID does not exist" containerID="907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db" Jul 02 23:25:12.635828 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.635750 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db"} err="failed to get container status \"907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db\": rpc error: code = NotFound desc = could not find container \"907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db\": container with ID starting with 907ce40aabb0f676029bb1850e9b9c80ea323b972dc9e7065cd734afce6b13db not found: ID does not exist" Jul 02 23:25:12.635828 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.635765 2555 scope.go:117] "RemoveContainer" containerID="df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939" Jul 02 23:25:12.635986 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:25:12.635971 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939\": container with ID starting with df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939 not found: ID does not exist" containerID="df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939" Jul 02 23:25:12.636028 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.635994 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939"} err="failed to get container status \"df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939\": rpc error: code = NotFound desc = could not find container \"df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939\": container with ID starting with df1785ab704fe6f3bc21151e96f26bf5a67a5aa1918ddb4f0c1f4ebf2ce68939 not found: ID does not exist" Jul 02 23:25:12.637901 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.637883 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:25:12.638148 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638136 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="alertmanager" Jul 02 23:25:12.638191 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638152 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="alertmanager" Jul 02 23:25:12.638191 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638168 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy-web" Jul 02 23:25:12.638191 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638174 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy-web" Jul 02 23:25:12.638191 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638182 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy-metric" Jul 02 23:25:12.638191 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638188 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy-metric" Jul 02 23:25:12.638191 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638193 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="init-config-reloader" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638199 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="init-config-reloader" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638212 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638219 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638227 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a453dd1-7520-48ed-a232-b43bdd78f854" containerName="console" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638232 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a453dd1-7520-48ed-a232-b43bdd78f854" containerName="console" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638237 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="prom-label-proxy" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638243 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="prom-label-proxy" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638251 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="config-reloader" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638256 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="config-reloader" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638303 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="config-reloader" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638312 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy-web" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638317 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a453dd1-7520-48ed-a232-b43bdd78f854" containerName="console" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638322 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="alertmanager" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638327 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy-metric" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638333 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="kube-rbac-proxy" Jul 02 23:25:12.638429 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.638337 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" containerName="prom-label-proxy" Jul 02 23:25:12.642906 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.642890 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.645175 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.645158 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 23:25:12.645308 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.645290 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 23:25:12.645445 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.645427 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-szlp8\"" Jul 02 23:25:12.645492 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.645434 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 23:25:12.646716 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.645981 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 23:25:12.646716 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.646038 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 23:25:12.646716 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.646047 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 23:25:12.648853 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.646832 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 23:25:12.648853 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.646909 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 23:25:12.654625 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.654603 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 23:25:12.655977 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.655958 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:25:12.657664 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657642 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657736 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b579166-b977-4c2d-9eb9-250e2748c634-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657736 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657708 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b579166-b977-4c2d-9eb9-250e2748c634-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657756 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-web-config\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657784 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b579166-b977-4c2d-9eb9-250e2748c634-config-out\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657807 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657901 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657823 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b579166-b977-4c2d-9eb9-250e2748c634-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657901 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657849 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657957 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657898 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657957 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657923 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.657957 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657952 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b579166-b977-4c2d-9eb9-250e2748c634-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.658065 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fpvv\" (UniqueName: \"kubernetes.io/projected/5b579166-b977-4c2d-9eb9-250e2748c634-kube-api-access-2fpvv\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.658065 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.657990 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759159 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759136 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b579166-b977-4c2d-9eb9-250e2748c634-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759163 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-web-config\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759182 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b579166-b977-4c2d-9eb9-250e2748c634-config-out\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759197 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759243 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759219 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b579166-b977-4c2d-9eb9-250e2748c634-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759447 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759253 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759447 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759407 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759541 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759541 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759494 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b579166-b977-4c2d-9eb9-250e2748c634-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759541 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759519 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fpvv\" (UniqueName: \"kubernetes.io/projected/5b579166-b977-4c2d-9eb9-250e2748c634-kube-api-access-2fpvv\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759694 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759553 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759694 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759601 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.759694 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.759635 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b579166-b977-4c2d-9eb9-250e2748c634-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.760399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.760342 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b579166-b977-4c2d-9eb9-250e2748c634-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.760640 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.760444 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b579166-b977-4c2d-9eb9-250e2748c634-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.762377 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.762336 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b579166-b977-4c2d-9eb9-250e2748c634-config-out\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.762467 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.762415 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.762526 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.762513 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-web-config\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.762716 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.762687 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.762816 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.762745 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.762816 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.762793 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b579166-b977-4c2d-9eb9-250e2748c634-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.762901 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.762833 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b579166-b977-4c2d-9eb9-250e2748c634-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.762985 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.762968 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.763111 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.763094 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.764385 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.764344 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b579166-b977-4c2d-9eb9-250e2748c634-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.767842 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.767821 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fpvv\" (UniqueName: \"kubernetes.io/projected/5b579166-b977-4c2d-9eb9-250e2748c634-kube-api-access-2fpvv\") pod \"alertmanager-main-0\" (UID: \"5b579166-b977-4c2d-9eb9-250e2748c634\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:12.954648 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:12.954627 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 23:25:13.086765 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:13.086743 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 23:25:13.088613 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:25:13.088586 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b579166_b977_4c2d_9eb9_250e2748c634.slice/crio-440e6ed568230023fdd04c7f4865f91b87ebdff4ec156f0ca26f45750e4ce6a0 WatchSource:0}: Error finding container 440e6ed568230023fdd04c7f4865f91b87ebdff4ec156f0ca26f45750e4ce6a0: Status 404 returned error can't find the container with id 440e6ed568230023fdd04c7f4865f91b87ebdff4ec156f0ca26f45750e4ce6a0 Jul 02 23:25:13.587558 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:13.587532 2555 generic.go:356] "Generic (PLEG): container finished" podID="5b579166-b977-4c2d-9eb9-250e2748c634" containerID="a6fcfa3da254bd9735e366d7a40803ca0db7bee1a683ae0837be161a97575820" exitCode=0 Jul 02 23:25:13.587969 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:13.587627 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b579166-b977-4c2d-9eb9-250e2748c634","Type":"ContainerDied","Data":"a6fcfa3da254bd9735e366d7a40803ca0db7bee1a683ae0837be161a97575820"} Jul 02 23:25:13.587969 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:13.587718 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b579166-b977-4c2d-9eb9-250e2748c634","Type":"ContainerStarted","Data":"440e6ed568230023fdd04c7f4865f91b87ebdff4ec156f0ca26f45750e4ce6a0"} Jul 02 23:25:13.682015 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:13.681982 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ed16a19-f3d1-43d8-8564-a22d8d4ad13a" path="/var/lib/kubelet/pods/5ed16a19-f3d1-43d8-8564-a22d8d4ad13a/volumes" Jul 02 23:25:14.595066 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.595034 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b579166-b977-4c2d-9eb9-250e2748c634","Type":"ContainerStarted","Data":"2225b81b2c9394c02ab71601ad3844d9ba4a8c705885c9b31b4783b8db69ab8b"} Jul 02 23:25:14.595470 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.595131 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b579166-b977-4c2d-9eb9-250e2748c634","Type":"ContainerStarted","Data":"ba2ef0e82d592bd4f1861162c821f50f408d4883d45b643bd8c330fb3dbf8771"} Jul 02 23:25:14.595470 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.595158 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b579166-b977-4c2d-9eb9-250e2748c634","Type":"ContainerStarted","Data":"35a977b8e98dc54d957f2c9cc9765bc111f5650f5a898537cb250d4fe6cfce15"} Jul 02 23:25:14.595470 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.595181 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b579166-b977-4c2d-9eb9-250e2748c634","Type":"ContainerStarted","Data":"6d076d977fd851820cdb7371ea6b2d197ff233324576b7ea731ce7fd3a3706b0"} Jul 02 23:25:14.595470 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.595204 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b579166-b977-4c2d-9eb9-250e2748c634","Type":"ContainerStarted","Data":"5c4076af46970851d69985ea4fee93add1beff7c298cd9bc67742657f484bb05"} Jul 02 23:25:14.595470 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.595226 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b579166-b977-4c2d-9eb9-250e2748c634","Type":"ContainerStarted","Data":"05b300522fe2578a9b18644c9b69458cccbb6615a07baa360f29103ac73e5e85"} Jul 02 23:25:14.620385 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.620323 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.605034655 podStartE2EDuration="2.620306223s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.271568ms" isStatefulPod=false podCreationTimestamp="2026-07-02 23:25:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:25:14.619355196 +0000 UTC m=+161.358647651" watchObservedRunningTime="2026-07-02 23:25:14.620306223 +0000 UTC m=+161.359598684" Jul 02 23:25:14.886930 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.886864 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55b9649fcd-v6x5t"] Jul 02 23:25:14.890306 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.890289 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:14.892711 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.892688 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 02 23:25:14.892817 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.892689 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 02 23:25:14.893354 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.893337 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 02 23:25:14.893454 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.893339 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 02 23:25:14.893506 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.893454 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mxltf\"" Jul 02 23:25:14.893659 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.893642 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 02 23:25:14.897655 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.897635 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 02 23:25:14.901479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.901458 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55b9649fcd-v6x5t"] Jul 02 23:25:14.974299 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.974271 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:14.974406 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.974304 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-federate-client-tls\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:14.974406 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.974321 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-metrics-client-ca\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:14.974406 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.974345 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99vtp\" (UniqueName: \"kubernetes.io/projected/31093d2e-124f-44be-a7d4-10dc1259a965-kube-api-access-99vtp\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:14.974406 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.974401 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-serving-certs-ca-bundle\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:14.974603 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.974428 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:14.974603 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.974484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-telemeter-client-tls\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:14.974603 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:14.974521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-secret-telemeter-client\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.074891 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.074863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99vtp\" (UniqueName: \"kubernetes.io/projected/31093d2e-124f-44be-a7d4-10dc1259a965-kube-api-access-99vtp\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.074983 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.074897 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-serving-certs-ca-bundle\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.074983 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.074932 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.074983 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.074958 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-telemeter-client-tls\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.074983 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.074976 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-secret-telemeter-client\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.075180 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.075003 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.075180 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.075026 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-federate-client-tls\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.075311 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.075288 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-metrics-client-ca\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.075743 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.075691 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-serving-certs-ca-bundle\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.076201 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.076178 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-metrics-client-ca\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.076279 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.076230 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31093d2e-124f-44be-a7d4-10dc1259a965-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.077682 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.077657 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-telemeter-client-tls\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.077761 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.077713 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-federate-client-tls\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.077811 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.077791 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.077848 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.077829 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/31093d2e-124f-44be-a7d4-10dc1259a965-secret-telemeter-client\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.082490 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.082469 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99vtp\" (UniqueName: \"kubernetes.io/projected/31093d2e-124f-44be-a7d4-10dc1259a965-kube-api-access-99vtp\") pod \"telemeter-client-55b9649fcd-v6x5t\" (UID: \"31093d2e-124f-44be-a7d4-10dc1259a965\") " pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.200880 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.200833 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" Jul 02 23:25:15.329827 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.329804 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55b9649fcd-v6x5t"] Jul 02 23:25:15.331451 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:25:15.331425 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31093d2e_124f_44be_a7d4_10dc1259a965.slice/crio-23a24c1f018566a6d525f070fcce3bb781ce98df15457405acb7eae7d5a4786d WatchSource:0}: Error finding container 23a24c1f018566a6d525f070fcce3bb781ce98df15457405acb7eae7d5a4786d: Status 404 returned error can't find the container with id 23a24c1f018566a6d525f070fcce3bb781ce98df15457405acb7eae7d5a4786d Jul 02 23:25:15.600607 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:15.600580 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" event={"ID":"31093d2e-124f-44be-a7d4-10dc1259a965","Type":"ContainerStarted","Data":"23a24c1f018566a6d525f070fcce3bb781ce98df15457405acb7eae7d5a4786d"} Jul 02 23:25:17.618651 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:17.618618 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" event={"ID":"31093d2e-124f-44be-a7d4-10dc1259a965","Type":"ContainerStarted","Data":"d0190e6cc3222bf1d1eb94e8bde27db529174dc96cdd76eb457320fe0b2d3cef"} Jul 02 23:25:17.619093 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:17.618682 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" event={"ID":"31093d2e-124f-44be-a7d4-10dc1259a965","Type":"ContainerStarted","Data":"9e94f315316174ae986193266bf3c4cc41e7f9cb0620c17bc409769e39f70102"} Jul 02 23:25:17.619093 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:17.618713 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" event={"ID":"31093d2e-124f-44be-a7d4-10dc1259a965","Type":"ContainerStarted","Data":"89a7e8681089397e96c89d1a71577601bc812323a1ca1bc73ac77908da350344"} Jul 02 23:25:17.639213 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:25:17.639172 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55b9649fcd-v6x5t" podStartSLOduration=1.56941733 podStartE2EDuration="3.639158325s" totalImagesPullingTime="2.069740995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:25:17.638668444 +0000 UTC m=+164.377960941" watchObservedRunningTime="2026-07-02 23:25:17.639158325 +0000 UTC m=+164.378450795" Jul 02 23:26:29.619574 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.619532 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57766c8c8-vlwhr"] Jul 02 23:26:29.622758 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.622742 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.625159 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.625132 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 23:26:29.625295 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.625182 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 23:26:29.625295 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.625195 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 23:26:29.625445 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.625354 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fqzxt\"" Jul 02 23:26:29.625445 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.625432 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 23:26:29.625547 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.625453 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 23:26:29.631315 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.630846 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 23:26:29.632162 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.632143 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57766c8c8-vlwhr"] Jul 02 23:26:29.799247 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.799218 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-oauth-config\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.799401 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.799268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-oauth-serving-cert\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.799401 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.799337 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-serving-cert\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.799401 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.799358 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-service-ca\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.799570 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.799408 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-config\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.799570 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.799440 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-trusted-ca-bundle\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.799570 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.799468 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxwjq\" (UniqueName: \"kubernetes.io/projected/238462d8-0d15-41d3-aa9c-4c99c3fda863-kube-api-access-bxwjq\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.900056 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.900003 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-oauth-serving-cert\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.900056 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.900038 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-serving-cert\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.900056 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.900053 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-service-ca\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.900256 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.900072 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-config\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.900256 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.900094 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-trusted-ca-bundle\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.900256 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.900112 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxwjq\" (UniqueName: \"kubernetes.io/projected/238462d8-0d15-41d3-aa9c-4c99c3fda863-kube-api-access-bxwjq\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.900256 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.900160 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-oauth-config\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.901043 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.901018 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-config\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.901127 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.901043 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-oauth-serving-cert\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.901127 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.901048 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-trusted-ca-bundle\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.901127 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.901071 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-service-ca\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.902665 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.902642 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-serving-cert\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.902740 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.902723 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-oauth-config\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.907552 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.907536 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxwjq\" (UniqueName: \"kubernetes.io/projected/238462d8-0d15-41d3-aa9c-4c99c3fda863-kube-api-access-bxwjq\") pod \"console-57766c8c8-vlwhr\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:29.934522 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:29.934504 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:30.059494 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:30.059465 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57766c8c8-vlwhr"] Jul 02 23:26:30.061869 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:26:30.061834 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238462d8_0d15_41d3_aa9c_4c99c3fda863.slice/crio-ecaada73aee7f7ad998f0d3277879cec17499d7b7f5c6d58971c3fb807f748eb WatchSource:0}: Error finding container ecaada73aee7f7ad998f0d3277879cec17499d7b7f5c6d58971c3fb807f748eb: Status 404 returned error can't find the container with id ecaada73aee7f7ad998f0d3277879cec17499d7b7f5c6d58971c3fb807f748eb Jul 02 23:26:30.892205 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:30.892172 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57766c8c8-vlwhr" event={"ID":"238462d8-0d15-41d3-aa9c-4c99c3fda863","Type":"ContainerStarted","Data":"6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070"} Jul 02 23:26:30.892593 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:30.892233 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57766c8c8-vlwhr" event={"ID":"238462d8-0d15-41d3-aa9c-4c99c3fda863","Type":"ContainerStarted","Data":"ecaada73aee7f7ad998f0d3277879cec17499d7b7f5c6d58971c3fb807f748eb"} Jul 02 23:26:30.909027 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:30.908984 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57766c8c8-vlwhr" podStartSLOduration=1.908969789 podStartE2EDuration="1.908969789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:26:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:26:30.90678668 +0000 UTC m=+237.646079155" watchObservedRunningTime="2026-07-02 23:26:30.908969789 +0000 UTC m=+237.648262247" Jul 02 23:26:39.934965 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:39.934932 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:39.935396 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:39.934999 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:39.939568 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:39.939547 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:26:40.935587 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:26:40.935558 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:27:33.565352 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:33.565319 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:27:33.566412 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:33.566384 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:27:33.571797 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:33.571777 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 23:27:48.642796 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.642765 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j"] Jul 02 23:27:48.646003 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.645988 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" Jul 02 23:27:48.648374 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.648334 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 23:27:48.648492 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.648398 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 23:27:48.649171 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.649153 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-cbl7x\"" Jul 02 23:27:48.655053 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.655029 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j"] Jul 02 23:27:48.775885 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.775860 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/683b96d8-c78e-46b4-ae2c-447c90c7363a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vlq5j\" (UID: \"683b96d8-c78e-46b4-ae2c-447c90c7363a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" Jul 02 23:27:48.775975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.775900 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7z8b\" (UniqueName: \"kubernetes.io/projected/683b96d8-c78e-46b4-ae2c-447c90c7363a-kube-api-access-s7z8b\") pod \"cert-manager-cainjector-559cdd8c87-vlq5j\" (UID: \"683b96d8-c78e-46b4-ae2c-447c90c7363a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" Jul 02 23:27:48.876982 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.876951 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/683b96d8-c78e-46b4-ae2c-447c90c7363a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vlq5j\" (UID: \"683b96d8-c78e-46b4-ae2c-447c90c7363a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" Jul 02 23:27:48.877073 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.876997 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7z8b\" (UniqueName: \"kubernetes.io/projected/683b96d8-c78e-46b4-ae2c-447c90c7363a-kube-api-access-s7z8b\") pod \"cert-manager-cainjector-559cdd8c87-vlq5j\" (UID: \"683b96d8-c78e-46b4-ae2c-447c90c7363a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" Jul 02 23:27:48.884505 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.884482 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/683b96d8-c78e-46b4-ae2c-447c90c7363a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vlq5j\" (UID: \"683b96d8-c78e-46b4-ae2c-447c90c7363a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" Jul 02 23:27:48.884810 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.884788 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7z8b\" (UniqueName: \"kubernetes.io/projected/683b96d8-c78e-46b4-ae2c-447c90c7363a-kube-api-access-s7z8b\") pod \"cert-manager-cainjector-559cdd8c87-vlq5j\" (UID: \"683b96d8-c78e-46b4-ae2c-447c90c7363a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" Jul 02 23:27:48.956114 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:48.956062 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" Jul 02 23:27:49.084007 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:49.083985 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j"] Jul 02 23:27:49.086350 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:27:49.086322 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod683b96d8_c78e_46b4_ae2c_447c90c7363a.slice/crio-35140c4c4067322e71e04a75a136417f077258ac2fb1e30eca76c8d2b037a8ac WatchSource:0}: Error finding container 35140c4c4067322e71e04a75a136417f077258ac2fb1e30eca76c8d2b037a8ac: Status 404 returned error can't find the container with id 35140c4c4067322e71e04a75a136417f077258ac2fb1e30eca76c8d2b037a8ac Jul 02 23:27:49.088279 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:49.088262 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:27:49.183643 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:49.183619 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" event={"ID":"683b96d8-c78e-46b4-ae2c-447c90c7363a","Type":"ContainerStarted","Data":"35140c4c4067322e71e04a75a136417f077258ac2fb1e30eca76c8d2b037a8ac"} Jul 02 23:27:53.204135 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:53.204092 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" event={"ID":"683b96d8-c78e-46b4-ae2c-447c90c7363a","Type":"ContainerStarted","Data":"77627db520508c92349cb0aac621b1681cc952b7d60e449bf56d2dfffd13cc21"} Jul 02 23:27:53.217727 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:27:53.217638 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vlq5j" podStartSLOduration=1.760137885 podStartE2EDuration="5.217619597s" totalImagesPullingTime="3.457481712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:27:53.216915609 +0000 UTC m=+319.956208068" watchObservedRunningTime="2026-07-02 23:27:53.217619597 +0000 UTC m=+319.956912057" Jul 02 23:28:00.907834 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:00.907800 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl"] Jul 02 23:28:00.911068 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:00.911053 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" Jul 02 23:28:00.913761 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:00.913740 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 23:28:00.914685 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:00.914669 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-g6pzg\"" Jul 02 23:28:00.914739 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:00.914669 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:28:00.918501 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:00.918482 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl"] Jul 02 23:28:00.971789 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:00.971760 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pl2h\" (UniqueName: \"kubernetes.io/projected/eb7681ba-c381-4453-9f66-ccdf717bfff2-kube-api-access-5pl2h\") pod \"openshift-lws-operator-fd8ccff4c-hvwbl\" (UID: \"eb7681ba-c381-4453-9f66-ccdf717bfff2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" Jul 02 23:28:00.971910 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:00.971807 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7681ba-c381-4453-9f66-ccdf717bfff2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hvwbl\" (UID: \"eb7681ba-c381-4453-9f66-ccdf717bfff2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" Jul 02 23:28:01.072695 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:01.072670 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pl2h\" (UniqueName: \"kubernetes.io/projected/eb7681ba-c381-4453-9f66-ccdf717bfff2-kube-api-access-5pl2h\") pod \"openshift-lws-operator-fd8ccff4c-hvwbl\" (UID: \"eb7681ba-c381-4453-9f66-ccdf717bfff2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" Jul 02 23:28:01.072786 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:01.072705 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7681ba-c381-4453-9f66-ccdf717bfff2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hvwbl\" (UID: \"eb7681ba-c381-4453-9f66-ccdf717bfff2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" Jul 02 23:28:01.072991 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:01.072977 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7681ba-c381-4453-9f66-ccdf717bfff2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hvwbl\" (UID: \"eb7681ba-c381-4453-9f66-ccdf717bfff2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" Jul 02 23:28:01.080341 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:01.080315 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pl2h\" (UniqueName: \"kubernetes.io/projected/eb7681ba-c381-4453-9f66-ccdf717bfff2-kube-api-access-5pl2h\") pod \"openshift-lws-operator-fd8ccff4c-hvwbl\" (UID: \"eb7681ba-c381-4453-9f66-ccdf717bfff2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" Jul 02 23:28:01.220777 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:01.220695 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" Jul 02 23:28:01.353488 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:01.353465 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl"] Jul 02 23:28:01.355668 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:28:01.355636 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb7681ba_c381_4453_9f66_ccdf717bfff2.slice/crio-78a55403ef5a901cbef3cdf6b8dd09fe96184eef5b77ddb91ee2e0ddfe7f0562 WatchSource:0}: Error finding container 78a55403ef5a901cbef3cdf6b8dd09fe96184eef5b77ddb91ee2e0ddfe7f0562: Status 404 returned error can't find the container with id 78a55403ef5a901cbef3cdf6b8dd09fe96184eef5b77ddb91ee2e0ddfe7f0562 Jul 02 23:28:02.242560 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:02.242524 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" event={"ID":"eb7681ba-c381-4453-9f66-ccdf717bfff2","Type":"ContainerStarted","Data":"78a55403ef5a901cbef3cdf6b8dd09fe96184eef5b77ddb91ee2e0ddfe7f0562"} Jul 02 23:28:10.278819 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:10.278787 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" event={"ID":"eb7681ba-c381-4453-9f66-ccdf717bfff2","Type":"ContainerStarted","Data":"29c001c9ce126938f500cfe802ab5c83039ff70fd4ac735e8ec1e1ae061f752b"} Jul 02 23:28:10.294599 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:10.294545 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hvwbl" podStartSLOduration=2.439401619 podStartE2EDuration="10.294530061s" totalImagesPullingTime="7.855128442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:28:10.293532594 +0000 UTC m=+337.032825054" watchObservedRunningTime="2026-07-02 23:28:10.294530061 +0000 UTC m=+337.033822518" Jul 02 23:28:28.539994 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.539924 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm"] Jul 02 23:28:28.546466 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.546441 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.549102 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.549082 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 23:28:28.549238 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.549220 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h86qq\"" Jul 02 23:28:28.549348 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.549240 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 23:28:28.549493 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.549480 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 23:28:28.549557 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.549534 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 23:28:28.555791 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.555770 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm"] Jul 02 23:28:28.662575 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.662542 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/230f226f-a085-4b9c-94d1-6810b0c40c1c-manifests\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.662575 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.662574 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230f226f-a085-4b9c-94d1-6810b0c40c1c-tmp\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.662796 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.662594 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/230f226f-a085-4b9c-94d1-6810b0c40c1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.662796 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.662636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230f226f-a085-4b9c-94d1-6810b0c40c1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.662796 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.662682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb2hc\" (UniqueName: \"kubernetes.io/projected/230f226f-a085-4b9c-94d1-6810b0c40c1c-kube-api-access-pb2hc\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.763772 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.763740 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230f226f-a085-4b9c-94d1-6810b0c40c1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.763934 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.763776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb2hc\" (UniqueName: \"kubernetes.io/projected/230f226f-a085-4b9c-94d1-6810b0c40c1c-kube-api-access-pb2hc\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.763934 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.763889 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/230f226f-a085-4b9c-94d1-6810b0c40c1c-manifests\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.763934 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.763919 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230f226f-a085-4b9c-94d1-6810b0c40c1c-tmp\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.764109 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.763948 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/230f226f-a085-4b9c-94d1-6810b0c40c1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.764355 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.764335 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/230f226f-a085-4b9c-94d1-6810b0c40c1c-manifests\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.766440 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.766411 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230f226f-a085-4b9c-94d1-6810b0c40c1c-tmp\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.766553 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.766535 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230f226f-a085-4b9c-94d1-6810b0c40c1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.766628 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.766609 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/230f226f-a085-4b9c-94d1-6810b0c40c1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.771713 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.771695 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb2hc\" (UniqueName: \"kubernetes.io/projected/230f226f-a085-4b9c-94d1-6810b0c40c1c-kube-api-access-pb2hc\") pod \"opendatahub-operator-controller-manager-6f45646c76-l5nkm\" (UID: \"230f226f-a085-4b9c-94d1-6810b0c40c1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:28.858289 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:28.858204 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:29.025148 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:29.025097 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm"] Jul 02 23:28:29.027502 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:28:29.027472 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230f226f_a085_4b9c_94d1_6810b0c40c1c.slice/crio-ddc5a621a36117a86ec46864eeb1e05e5ed5202840df9e39ce4772e4cad201d3 WatchSource:0}: Error finding container ddc5a621a36117a86ec46864eeb1e05e5ed5202840df9e39ce4772e4cad201d3: Status 404 returned error can't find the container with id ddc5a621a36117a86ec46864eeb1e05e5ed5202840df9e39ce4772e4cad201d3 Jul 02 23:28:29.352710 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:29.352674 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" event={"ID":"230f226f-a085-4b9c-94d1-6810b0c40c1c","Type":"ContainerStarted","Data":"ddc5a621a36117a86ec46864eeb1e05e5ed5202840df9e39ce4772e4cad201d3"} Jul 02 23:28:32.370191 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:32.370160 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" event={"ID":"230f226f-a085-4b9c-94d1-6810b0c40c1c","Type":"ContainerStarted","Data":"719d88c932c970a95b485fdfd55da6ca445006236ae970df4e3411868756ad7d"} Jul 02 23:28:33.376637 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:33.376569 2555 generic.go:356] "Generic (PLEG): container finished" podID="230f226f-a085-4b9c-94d1-6810b0c40c1c" containerID="719d88c932c970a95b485fdfd55da6ca445006236ae970df4e3411868756ad7d" exitCode=0 Jul 02 23:28:33.376926 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:33.376636 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" event={"ID":"230f226f-a085-4b9c-94d1-6810b0c40c1c","Type":"ContainerDied","Data":"719d88c932c970a95b485fdfd55da6ca445006236ae970df4e3411868756ad7d"} Jul 02 23:28:34.383163 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:34.383123 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" event={"ID":"230f226f-a085-4b9c-94d1-6810b0c40c1c","Type":"ContainerStarted","Data":"45cae82dbfb45cc5829774617d75c256e430c65f6c2b55568384825a6bf3858d"} Jul 02 23:28:34.383606 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:34.383245 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:34.402580 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:34.402518 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" podStartSLOduration=2.100588204 podStartE2EDuration="6.402496492s" totalImagesPullingTime="3.247857869s" totalInitContainerRuntime="1.054050419s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:28:34.401598924 +0000 UTC m=+361.140891401" watchObservedRunningTime="2026-07-02 23:28:34.402496492 +0000 UTC m=+361.141788951" Jul 02 23:28:45.390468 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:45.390435 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-l5nkm" Jul 02 23:28:48.035419 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.035387 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-576b444b4-8p2gf"] Jul 02 23:28:48.043564 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.043543 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.046011 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.045989 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 23:28:48.046116 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.046060 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 23:28:48.046853 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.046831 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 23:28:48.046943 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.046926 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 23:28:48.048192 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.048168 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-48t4x\"" Jul 02 23:28:48.050125 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.050103 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-576b444b4-8p2gf"] Jul 02 23:28:48.132395 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.132356 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3f5372d-6d8e-4b44-bf5f-56f43829b580-tls-certs\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.132482 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.132425 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f5372d-6d8e-4b44-bf5f-56f43829b580-tmp\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.132522 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.132489 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhnm\" (UniqueName: \"kubernetes.io/projected/a3f5372d-6d8e-4b44-bf5f-56f43829b580-kube-api-access-9xhnm\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.233783 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.233754 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f5372d-6d8e-4b44-bf5f-56f43829b580-tmp\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.233899 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.233794 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhnm\" (UniqueName: \"kubernetes.io/projected/a3f5372d-6d8e-4b44-bf5f-56f43829b580-kube-api-access-9xhnm\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.233899 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.233849 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3f5372d-6d8e-4b44-bf5f-56f43829b580-tls-certs\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.236131 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.236110 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f5372d-6d8e-4b44-bf5f-56f43829b580-tmp\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.236616 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.236590 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3f5372d-6d8e-4b44-bf5f-56f43829b580-tls-certs\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.248147 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.248125 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhnm\" (UniqueName: \"kubernetes.io/projected/a3f5372d-6d8e-4b44-bf5f-56f43829b580-kube-api-access-9xhnm\") pod \"kube-auth-proxy-576b444b4-8p2gf\" (UID: \"a3f5372d-6d8e-4b44-bf5f-56f43829b580\") " pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.354917 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.354892 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" Jul 02 23:28:48.538685 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:48.538662 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-576b444b4-8p2gf"] Jul 02 23:28:48.540491 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:28:48.540463 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f5372d_6d8e_4b44_bf5f_56f43829b580.slice/crio-c53112338966539d869d5eb647ad79c9d709891b52fe229f0543b39fb527a01a WatchSource:0}: Error finding container c53112338966539d869d5eb647ad79c9d709891b52fe229f0543b39fb527a01a: Status 404 returned error can't find the container with id c53112338966539d869d5eb647ad79c9d709891b52fe229f0543b39fb527a01a Jul 02 23:28:49.455098 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:49.455058 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" event={"ID":"a3f5372d-6d8e-4b44-bf5f-56f43829b580","Type":"ContainerStarted","Data":"c53112338966539d869d5eb647ad79c9d709891b52fe229f0543b39fb527a01a"} Jul 02 23:28:51.466399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:51.466296 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" event={"ID":"a3f5372d-6d8e-4b44-bf5f-56f43829b580","Type":"ContainerStarted","Data":"189e1053b4d30976cc299ff8f1d0866deedb154e59b8f19adb1a98954579d92d"} Jul 02 23:28:51.484049 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:51.484007 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-576b444b4-8p2gf" podStartSLOduration=1.946431958 podStartE2EDuration="4.483992918s" totalImagesPullingTime="2.53756096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:28:51.483042963 +0000 UTC m=+378.222335422" watchObservedRunningTime="2026-07-02 23:28:51.483992918 +0000 UTC m=+378.223285375" Jul 02 23:28:56.140210 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.140173 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q"] Jul 02 23:28:56.143521 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.143504 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.148626 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.148604 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 23:28:56.148754 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.148687 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cppbc\"" Jul 02 23:28:56.148754 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.148710 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 23:28:56.148754 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.148693 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 23:28:56.163040 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.163021 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q"] Jul 02 23:28:56.312882 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.312852 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rsv6\" (UniqueName: \"kubernetes.io/projected/821ce603-37b3-4e65-b559-83f2ec1b7ef2-kube-api-access-9rsv6\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.313068 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.312901 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821ce603-37b3-4e65-b559-83f2ec1b7ef2-cert\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.313068 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.312947 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/821ce603-37b3-4e65-b559-83f2ec1b7ef2-metrics-cert\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.313068 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.312971 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/821ce603-37b3-4e65-b559-83f2ec1b7ef2-manager-config\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.414215 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.414113 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rsv6\" (UniqueName: \"kubernetes.io/projected/821ce603-37b3-4e65-b559-83f2ec1b7ef2-kube-api-access-9rsv6\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.414215 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.414185 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821ce603-37b3-4e65-b559-83f2ec1b7ef2-cert\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.414215 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.414204 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/821ce603-37b3-4e65-b559-83f2ec1b7ef2-metrics-cert\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.414533 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.414227 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/821ce603-37b3-4e65-b559-83f2ec1b7ef2-manager-config\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.414906 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.414882 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/821ce603-37b3-4e65-b559-83f2ec1b7ef2-manager-config\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.416855 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.416829 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/821ce603-37b3-4e65-b559-83f2ec1b7ef2-metrics-cert\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.416941 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.416877 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821ce603-37b3-4e65-b559-83f2ec1b7ef2-cert\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.421786 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.421766 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rsv6\" (UniqueName: \"kubernetes.io/projected/821ce603-37b3-4e65-b559-83f2ec1b7ef2-kube-api-access-9rsv6\") pod \"lws-controller-manager-679bd8f54d-mt42q\" (UID: \"821ce603-37b3-4e65-b559-83f2ec1b7ef2\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.452757 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.452736 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:56.590832 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:56.590805 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q"] Jul 02 23:28:56.592949 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:28:56.592917 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821ce603_37b3_4e65_b559_83f2ec1b7ef2.slice/crio-626c73400eaa64fee947736f329df84ceefd0293a671bc7cec74c9a3bb9e7ed2 WatchSource:0}: Error finding container 626c73400eaa64fee947736f329df84ceefd0293a671bc7cec74c9a3bb9e7ed2: Status 404 returned error can't find the container with id 626c73400eaa64fee947736f329df84ceefd0293a671bc7cec74c9a3bb9e7ed2 Jul 02 23:28:57.496492 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:57.496447 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" event={"ID":"821ce603-37b3-4e65-b559-83f2ec1b7ef2","Type":"ContainerStarted","Data":"626c73400eaa64fee947736f329df84ceefd0293a671bc7cec74c9a3bb9e7ed2"} Jul 02 23:28:58.502424 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:58.502392 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" event={"ID":"821ce603-37b3-4e65-b559-83f2ec1b7ef2","Type":"ContainerStarted","Data":"fcde92039214e4d088bda13a2b61d3a6e67ac72b65b4dccb29541205ee91734b"} Jul 02 23:28:58.502770 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:58.502530 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:28:58.538449 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:28:58.538396 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" podStartSLOduration=0.811461441 podStartE2EDuration="2.538379798s" totalImagesPullingTime="1.726918357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:28:58.536730791 +0000 UTC m=+385.276023250" watchObservedRunningTime="2026-07-02 23:28:58.538379798 +0000 UTC m=+385.277672247" Jul 02 23:29:09.509905 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:29:09.509873 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-mt42q" Jul 02 23:30:38.818666 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:38.818630 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57766c8c8-vlwhr"] Jul 02 23:30:41.212496 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.212460 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6"] Jul 02 23:30:41.215989 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.215970 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.218564 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.218532 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 02 23:30:41.218564 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.218529 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:30:41.219336 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.219320 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 02 23:30:41.219437 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.219398 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-svhmc\"" Jul 02 23:30:41.219437 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.219427 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:30:41.223913 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.223893 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6"] Jul 02 23:30:41.352820 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.352780 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zpvh\" (UniqueName: \"kubernetes.io/projected/e53ff879-1c33-4789-bedc-603bc7fc8d7b-kube-api-access-6zpvh\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.352995 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.352840 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e53ff879-1c33-4789-bedc-603bc7fc8d7b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.352995 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.352862 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53ff879-1c33-4789-bedc-603bc7fc8d7b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.454374 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.454332 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e53ff879-1c33-4789-bedc-603bc7fc8d7b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.454541 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.454423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53ff879-1c33-4789-bedc-603bc7fc8d7b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.454584 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.454541 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zpvh\" (UniqueName: \"kubernetes.io/projected/e53ff879-1c33-4789-bedc-603bc7fc8d7b-kube-api-access-6zpvh\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.454584 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:30:41.454570 2555 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 02 23:30:41.454689 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:30:41.454677 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53ff879-1c33-4789-bedc-603bc7fc8d7b-plugin-serving-cert podName:e53ff879-1c33-4789-bedc-603bc7fc8d7b nodeName:}" failed. No retries permitted until 2026-07-02 23:30:41.954659359 +0000 UTC m=+488.693951815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e53ff879-1c33-4789-bedc-603bc7fc8d7b-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-zc4l6" (UID: "e53ff879-1c33-4789-bedc-603bc7fc8d7b") : secret "plugin-serving-cert" not found Jul 02 23:30:41.454950 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.454924 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e53ff879-1c33-4789-bedc-603bc7fc8d7b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.463520 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.463445 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zpvh\" (UniqueName: \"kubernetes.io/projected/e53ff879-1c33-4789-bedc-603bc7fc8d7b-kube-api-access-6zpvh\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.959268 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.959234 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53ff879-1c33-4789-bedc-603bc7fc8d7b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:41.961822 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:41.961793 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53ff879-1c33-4789-bedc-603bc7fc8d7b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zc4l6\" (UID: \"e53ff879-1c33-4789-bedc-603bc7fc8d7b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:42.126300 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:42.126259 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" Jul 02 23:30:42.265895 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:42.265791 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6"] Jul 02 23:30:42.269318 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:30:42.269286 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode53ff879_1c33_4789_bedc_603bc7fc8d7b.slice/crio-b222b5acf284dcd09ff20f97f2080671f4ae840b8e25ccd7c13ff9a4db009a44 WatchSource:0}: Error finding container b222b5acf284dcd09ff20f97f2080671f4ae840b8e25ccd7c13ff9a4db009a44: Status 404 returned error can't find the container with id b222b5acf284dcd09ff20f97f2080671f4ae840b8e25ccd7c13ff9a4db009a44 Jul 02 23:30:42.970529 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:30:42.970490 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" event={"ID":"e53ff879-1c33-4789-bedc-603bc7fc8d7b","Type":"ContainerStarted","Data":"b222b5acf284dcd09ff20f97f2080671f4ae840b8e25ccd7c13ff9a4db009a44"} Jul 02 23:31:03.839157 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:03.839088 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57766c8c8-vlwhr" podUID="238462d8-0d15-41d3-aa9c-4c99c3fda863" containerName="console" containerID="cri-o://6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070" gracePeriod=15 Jul 02 23:31:04.673479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.673441 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:31:04.775392 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775340 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-config\") pod \"238462d8-0d15-41d3-aa9c-4c99c3fda863\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " Jul 02 23:31:04.775528 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775430 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxwjq\" (UniqueName: \"kubernetes.io/projected/238462d8-0d15-41d3-aa9c-4c99c3fda863-kube-api-access-bxwjq\") pod \"238462d8-0d15-41d3-aa9c-4c99c3fda863\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " Jul 02 23:31:04.775528 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775457 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-oauth-serving-cert\") pod \"238462d8-0d15-41d3-aa9c-4c99c3fda863\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " Jul 02 23:31:04.775528 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775494 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-trusted-ca-bundle\") pod \"238462d8-0d15-41d3-aa9c-4c99c3fda863\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " Jul 02 23:31:04.775771 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775561 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-oauth-config\") pod \"238462d8-0d15-41d3-aa9c-4c99c3fda863\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " Jul 02 23:31:04.775771 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775582 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-serving-cert\") pod \"238462d8-0d15-41d3-aa9c-4c99c3fda863\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " Jul 02 23:31:04.775771 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775609 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-service-ca\") pod \"238462d8-0d15-41d3-aa9c-4c99c3fda863\" (UID: \"238462d8-0d15-41d3-aa9c-4c99c3fda863\") " Jul 02 23:31:04.775771 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775711 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-config" (OuterVolumeSpecName: "console-config") pod "238462d8-0d15-41d3-aa9c-4c99c3fda863" (UID: "238462d8-0d15-41d3-aa9c-4c99c3fda863"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:31:04.775979 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775951 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "238462d8-0d15-41d3-aa9c-4c99c3fda863" (UID: "238462d8-0d15-41d3-aa9c-4c99c3fda863"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:31:04.775979 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775965 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "238462d8-0d15-41d3-aa9c-4c99c3fda863" (UID: "238462d8-0d15-41d3-aa9c-4c99c3fda863"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:31:04.776083 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775985 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-config\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:31:04.776083 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.775986 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-service-ca" (OuterVolumeSpecName: "service-ca") pod "238462d8-0d15-41d3-aa9c-4c99c3fda863" (UID: "238462d8-0d15-41d3-aa9c-4c99c3fda863"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:31:04.777882 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.777852 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "238462d8-0d15-41d3-aa9c-4c99c3fda863" (UID: "238462d8-0d15-41d3-aa9c-4c99c3fda863"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:31:04.778001 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.777971 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "238462d8-0d15-41d3-aa9c-4c99c3fda863" (UID: "238462d8-0d15-41d3-aa9c-4c99c3fda863"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:31:04.778001 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.777987 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/238462d8-0d15-41d3-aa9c-4c99c3fda863-kube-api-access-bxwjq" (OuterVolumeSpecName: "kube-api-access-bxwjq") pod "238462d8-0d15-41d3-aa9c-4c99c3fda863" (UID: "238462d8-0d15-41d3-aa9c-4c99c3fda863"). InnerVolumeSpecName "kube-api-access-bxwjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:31:04.877152 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.877109 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxwjq\" (UniqueName: \"kubernetes.io/projected/238462d8-0d15-41d3-aa9c-4c99c3fda863-kube-api-access-bxwjq\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:31:04.877152 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.877145 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-oauth-serving-cert\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:31:04.877152 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.877156 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-trusted-ca-bundle\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:31:04.877622 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.877166 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-oauth-config\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:31:04.877622 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.877174 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/238462d8-0d15-41d3-aa9c-4c99c3fda863-console-serving-cert\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:31:04.877622 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:04.877182 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/238462d8-0d15-41d3-aa9c-4c99c3fda863-service-ca\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:31:05.095623 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.095588 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" event={"ID":"e53ff879-1c33-4789-bedc-603bc7fc8d7b","Type":"ContainerStarted","Data":"cde57c2dffbd723db2381451a3988d49aad3d330fc466effa264cc654baf9487"} Jul 02 23:31:05.096839 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.096814 2555 generic.go:356] "Generic (PLEG): container finished" podID="238462d8-0d15-41d3-aa9c-4c99c3fda863" containerID="6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070" exitCode=0 Jul 02 23:31:05.096989 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.096856 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57766c8c8-vlwhr" Jul 02 23:31:05.096989 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.096898 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57766c8c8-vlwhr" event={"ID":"238462d8-0d15-41d3-aa9c-4c99c3fda863","Type":"ContainerDied","Data":"6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070"} Jul 02 23:31:05.096989 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.096951 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57766c8c8-vlwhr" event={"ID":"238462d8-0d15-41d3-aa9c-4c99c3fda863","Type":"ContainerDied","Data":"ecaada73aee7f7ad998f0d3277879cec17499d7b7f5c6d58971c3fb807f748eb"} Jul 02 23:31:05.096989 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.096972 2555 scope.go:117] "RemoveContainer" containerID="6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070" Jul 02 23:31:05.106444 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.106423 2555 scope.go:117] "RemoveContainer" containerID="6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070" Jul 02 23:31:05.106748 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:31:05.106728 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070\": container with ID starting with 6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070 not found: ID does not exist" containerID="6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070" Jul 02 23:31:05.106809 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.106763 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070"} err="failed to get container status \"6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070\": rpc error: code = NotFound desc = could not find container \"6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070\": container with ID starting with 6a04a30e6041513bfd78481b3dbd3626ac278ffa7c7bfae50f829cec80228070 not found: ID does not exist" Jul 02 23:31:05.110813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.110770 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zc4l6" podStartSLOduration=1.676329107 podStartE2EDuration="24.110755588s" totalImagesPullingTime="22.434426481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:31:05.109674315 +0000 UTC m=+511.848966772" watchObservedRunningTime="2026-07-02 23:31:05.110755588 +0000 UTC m=+511.850048044" Jul 02 23:31:05.127117 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.127086 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57766c8c8-vlwhr"] Jul 02 23:31:05.132558 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.132527 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57766c8c8-vlwhr"] Jul 02 23:31:05.683289 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:05.683207 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="238462d8-0d15-41d3-aa9c-4c99c3fda863" path="/var/lib/kubelet/pods/238462d8-0d15-41d3-aa9c-4c99c3fda863/volumes" Jul 02 23:31:39.012598 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.012555 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:31:39.013016 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.012902 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238462d8-0d15-41d3-aa9c-4c99c3fda863" containerName="console" Jul 02 23:31:39.013016 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.012913 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="238462d8-0d15-41d3-aa9c-4c99c3fda863" containerName="console" Jul 02 23:31:39.013016 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.012988 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="238462d8-0d15-41d3-aa9c-4c99c3fda863" containerName="console" Jul 02 23:31:39.074137 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.074051 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:31:39.074137 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.074135 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:39.074344 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.074166 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:31:39.076450 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.076432 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 23:31:39.164947 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.164907 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4854edda-01da-493c-b0bd-d6735e5524bc-config-file\") pod \"limitador-limitador-54d6d988d5-xdz5w\" (UID: \"4854edda-01da-493c-b0bd-d6735e5524bc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:39.165127 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.164965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vp4f\" (UniqueName: \"kubernetes.io/projected/4854edda-01da-493c-b0bd-d6735e5524bc-kube-api-access-6vp4f\") pod \"limitador-limitador-54d6d988d5-xdz5w\" (UID: \"4854edda-01da-493c-b0bd-d6735e5524bc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:39.265586 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.265481 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4854edda-01da-493c-b0bd-d6735e5524bc-config-file\") pod \"limitador-limitador-54d6d988d5-xdz5w\" (UID: \"4854edda-01da-493c-b0bd-d6735e5524bc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:39.265586 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.265571 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vp4f\" (UniqueName: \"kubernetes.io/projected/4854edda-01da-493c-b0bd-d6735e5524bc-kube-api-access-6vp4f\") pod \"limitador-limitador-54d6d988d5-xdz5w\" (UID: \"4854edda-01da-493c-b0bd-d6735e5524bc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:39.266163 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.266116 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4854edda-01da-493c-b0bd-d6735e5524bc-config-file\") pod \"limitador-limitador-54d6d988d5-xdz5w\" (UID: \"4854edda-01da-493c-b0bd-d6735e5524bc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:39.273526 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.273500 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vp4f\" (UniqueName: \"kubernetes.io/projected/4854edda-01da-493c-b0bd-d6735e5524bc-kube-api-access-6vp4f\") pod \"limitador-limitador-54d6d988d5-xdz5w\" (UID: \"4854edda-01da-493c-b0bd-d6735e5524bc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:39.385313 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.385270 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:39.515944 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.515872 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-pcqzr"] Jul 02 23:31:39.520217 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.520197 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pcqzr" Jul 02 23:31:39.522595 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.522569 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gmx56\"" Jul 02 23:31:39.525199 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.525173 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pcqzr"] Jul 02 23:31:39.535670 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.535637 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:31:39.539779 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:31:39.539744 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4854edda_01da_493c_b0bd_d6735e5524bc.slice/crio-95f69877aea169b499672a1255058392bdba82c049b39e512997498ec903a7f6 WatchSource:0}: Error finding container 95f69877aea169b499672a1255058392bdba82c049b39e512997498ec903a7f6: Status 404 returned error can't find the container with id 95f69877aea169b499672a1255058392bdba82c049b39e512997498ec903a7f6 Jul 02 23:31:39.669713 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.669676 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwcxk\" (UniqueName: \"kubernetes.io/projected/42474a58-6ccb-4a38-8cd3-ec6cdbd52954-kube-api-access-xwcxk\") pod \"authorino-75d556bd77-pcqzr\" (UID: \"42474a58-6ccb-4a38-8cd3-ec6cdbd52954\") " pod="kuadrant-system/authorino-75d556bd77-pcqzr" Jul 02 23:31:39.770813 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.770714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwcxk\" (UniqueName: \"kubernetes.io/projected/42474a58-6ccb-4a38-8cd3-ec6cdbd52954-kube-api-access-xwcxk\") pod \"authorino-75d556bd77-pcqzr\" (UID: \"42474a58-6ccb-4a38-8cd3-ec6cdbd52954\") " pod="kuadrant-system/authorino-75d556bd77-pcqzr" Jul 02 23:31:39.778588 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.778562 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwcxk\" (UniqueName: \"kubernetes.io/projected/42474a58-6ccb-4a38-8cd3-ec6cdbd52954-kube-api-access-xwcxk\") pod \"authorino-75d556bd77-pcqzr\" (UID: \"42474a58-6ccb-4a38-8cd3-ec6cdbd52954\") " pod="kuadrant-system/authorino-75d556bd77-pcqzr" Jul 02 23:31:39.830875 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.830833 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pcqzr" Jul 02 23:31:39.965789 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:39.965759 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pcqzr"] Jul 02 23:31:39.968206 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:31:39.968171 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42474a58_6ccb_4a38_8cd3_ec6cdbd52954.slice/crio-7ea934bc334b84b49112dfe9d28902ae9aebb7431ddcf2761417890dc70d8491 WatchSource:0}: Error finding container 7ea934bc334b84b49112dfe9d28902ae9aebb7431ddcf2761417890dc70d8491: Status 404 returned error can't find the container with id 7ea934bc334b84b49112dfe9d28902ae9aebb7431ddcf2761417890dc70d8491 Jul 02 23:31:40.280788 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:40.280695 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pcqzr" event={"ID":"42474a58-6ccb-4a38-8cd3-ec6cdbd52954","Type":"ContainerStarted","Data":"7ea934bc334b84b49112dfe9d28902ae9aebb7431ddcf2761417890dc70d8491"} Jul 02 23:31:40.283415 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:40.283291 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" event={"ID":"4854edda-01da-493c-b0bd-d6735e5524bc","Type":"ContainerStarted","Data":"95f69877aea169b499672a1255058392bdba82c049b39e512997498ec903a7f6"} Jul 02 23:31:44.306719 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:44.306684 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pcqzr" event={"ID":"42474a58-6ccb-4a38-8cd3-ec6cdbd52954","Type":"ContainerStarted","Data":"bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f"} Jul 02 23:31:44.308176 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:44.308150 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" event={"ID":"4854edda-01da-493c-b0bd-d6735e5524bc","Type":"ContainerStarted","Data":"dcb0edbc5daab023f7051a9e8e5ab49e616712a79b89fad1de6d355640d3f672"} Jul 02 23:31:44.308277 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:44.308202 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:31:44.320797 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:44.320752 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-pcqzr" podStartSLOduration=1.258086047 podStartE2EDuration="5.320735093s" totalImagesPullingTime="4.062649046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:31:44.31875181 +0000 UTC m=+551.058044269" watchObservedRunningTime="2026-07-02 23:31:44.320735093 +0000 UTC m=+551.060027550" Jul 02 23:31:44.337301 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:44.337250 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" podStartSLOduration=1.853125569 podStartE2EDuration="6.337237369s" totalImagesPullingTime="4.4841118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:31:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:31:44.336638359 +0000 UTC m=+551.075930844" watchObservedRunningTime="2026-07-02 23:31:44.337237369 +0000 UTC m=+551.076529827" Jul 02 23:31:55.314338 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:31:55.314308 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xdz5w" Jul 02 23:32:10.818436 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:10.818392 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whvns"] Jul 02 23:32:10.824852 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:10.824817 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-whvns" Jul 02 23:32:10.829221 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:10.829185 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whvns"] Jul 02 23:32:10.956521 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:10.956478 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7h47\" (UniqueName: \"kubernetes.io/projected/95108d71-2908-4a98-8a27-9b2ba48b2e2c-kube-api-access-b7h47\") pod \"authorino-5d4b67c446-whvns\" (UID: \"95108d71-2908-4a98-8a27-9b2ba48b2e2c\") " pod="kuadrant-system/authorino-5d4b67c446-whvns" Jul 02 23:32:10.996345 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:10.996315 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whvns"] Jul 02 23:32:10.996622 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:32:10.996602 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-b7h47], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-whvns" podUID="95108d71-2908-4a98-8a27-9b2ba48b2e2c" Jul 02 23:32:11.017509 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.017462 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-9mzsc"] Jul 02 23:32:11.025921 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.025882 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.027772 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.027744 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-9mzsc"] Jul 02 23:32:11.028399 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.028353 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 23:32:11.057580 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.057542 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7h47\" (UniqueName: \"kubernetes.io/projected/95108d71-2908-4a98-8a27-9b2ba48b2e2c-kube-api-access-b7h47\") pod \"authorino-5d4b67c446-whvns\" (UID: \"95108d71-2908-4a98-8a27-9b2ba48b2e2c\") " pod="kuadrant-system/authorino-5d4b67c446-whvns" Jul 02 23:32:11.065692 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.065655 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-9mzsc"] Jul 02 23:32:11.066030 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:32:11.065932 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ftsb6 tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-ftsb6 tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-9mzsc" podUID="f65805ca-7829-4bc4-afa9-9cd0fadabb71" Jul 02 23:32:11.067915 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.067890 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7h47\" (UniqueName: \"kubernetes.io/projected/95108d71-2908-4a98-8a27-9b2ba48b2e2c-kube-api-access-b7h47\") pod \"authorino-5d4b67c446-whvns\" (UID: \"95108d71-2908-4a98-8a27-9b2ba48b2e2c\") " pod="kuadrant-system/authorino-5d4b67c446-whvns" Jul 02 23:32:11.097078 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.096990 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-868f966f6-krb7k"] Jul 02 23:32:11.100966 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.100946 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:32:11.107532 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.107504 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-868f966f6-krb7k"] Jul 02 23:32:11.158449 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.158397 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f65805ca-7829-4bc4-afa9-9cd0fadabb71-tls-cert\") pod \"authorino-76877994-9mzsc\" (UID: \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\") " pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.158449 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.158451 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df119d6a-d8ca-4a50-a37d-366942fdc1c8-tls-cert\") pod \"authorino-868f966f6-krb7k\" (UID: \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\") " pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:32:11.158692 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.158498 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq5gz\" (UniqueName: \"kubernetes.io/projected/df119d6a-d8ca-4a50-a37d-366942fdc1c8-kube-api-access-gq5gz\") pod \"authorino-868f966f6-krb7k\" (UID: \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\") " pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:32:11.158692 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.158543 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftsb6\" (UniqueName: \"kubernetes.io/projected/f65805ca-7829-4bc4-afa9-9cd0fadabb71-kube-api-access-ftsb6\") pod \"authorino-76877994-9mzsc\" (UID: \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\") " pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.259174 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.259127 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f65805ca-7829-4bc4-afa9-9cd0fadabb71-tls-cert\") pod \"authorino-76877994-9mzsc\" (UID: \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\") " pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.259174 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.259170 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df119d6a-d8ca-4a50-a37d-366942fdc1c8-tls-cert\") pod \"authorino-868f966f6-krb7k\" (UID: \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\") " pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:32:11.259507 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.259193 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq5gz\" (UniqueName: \"kubernetes.io/projected/df119d6a-d8ca-4a50-a37d-366942fdc1c8-kube-api-access-gq5gz\") pod \"authorino-868f966f6-krb7k\" (UID: \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\") " pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:32:11.259507 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.259227 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftsb6\" (UniqueName: \"kubernetes.io/projected/f65805ca-7829-4bc4-afa9-9cd0fadabb71-kube-api-access-ftsb6\") pod \"authorino-76877994-9mzsc\" (UID: \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\") " pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.261908 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.261877 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f65805ca-7829-4bc4-afa9-9cd0fadabb71-tls-cert\") pod \"authorino-76877994-9mzsc\" (UID: \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\") " pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.262025 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.261952 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df119d6a-d8ca-4a50-a37d-366942fdc1c8-tls-cert\") pod \"authorino-868f966f6-krb7k\" (UID: \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\") " pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:32:11.268281 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.268245 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftsb6\" (UniqueName: \"kubernetes.io/projected/f65805ca-7829-4bc4-afa9-9cd0fadabb71-kube-api-access-ftsb6\") pod \"authorino-76877994-9mzsc\" (UID: \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\") " pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.268414 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.268352 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq5gz\" (UniqueName: \"kubernetes.io/projected/df119d6a-d8ca-4a50-a37d-366942fdc1c8-kube-api-access-gq5gz\") pod \"authorino-868f966f6-krb7k\" (UID: \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\") " pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:32:11.412182 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.412094 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:32:11.438316 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.438289 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-whvns" Jul 02 23:32:11.438489 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.438287 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.446557 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.446528 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-whvns" Jul 02 23:32:11.449828 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.449802 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:11.553983 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.553938 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-868f966f6-krb7k"] Jul 02 23:32:11.557433 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:32:11.557401 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf119d6a_d8ca_4a50_a37d_366942fdc1c8.slice/crio-dd957e9d22c3322fd2e264b65d06b217fd4745463845410f094bae89abc773f7 WatchSource:0}: Error finding container dd957e9d22c3322fd2e264b65d06b217fd4745463845410f094bae89abc773f7: Status 404 returned error can't find the container with id dd957e9d22c3322fd2e264b65d06b217fd4745463845410f094bae89abc773f7 Jul 02 23:32:11.562426 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.562405 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7h47\" (UniqueName: \"kubernetes.io/projected/95108d71-2908-4a98-8a27-9b2ba48b2e2c-kube-api-access-b7h47\") pod \"95108d71-2908-4a98-8a27-9b2ba48b2e2c\" (UID: \"95108d71-2908-4a98-8a27-9b2ba48b2e2c\") " Jul 02 23:32:11.562513 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.562443 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f65805ca-7829-4bc4-afa9-9cd0fadabb71-tls-cert\") pod \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\" (UID: \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\") " Jul 02 23:32:11.562574 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.562515 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftsb6\" (UniqueName: \"kubernetes.io/projected/f65805ca-7829-4bc4-afa9-9cd0fadabb71-kube-api-access-ftsb6\") pod \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\" (UID: \"f65805ca-7829-4bc4-afa9-9cd0fadabb71\") " Jul 02 23:32:11.564850 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.564810 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f65805ca-7829-4bc4-afa9-9cd0fadabb71-kube-api-access-ftsb6" (OuterVolumeSpecName: "kube-api-access-ftsb6") pod "f65805ca-7829-4bc4-afa9-9cd0fadabb71" (UID: "f65805ca-7829-4bc4-afa9-9cd0fadabb71"). InnerVolumeSpecName "kube-api-access-ftsb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:11.564850 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.564845 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65805ca-7829-4bc4-afa9-9cd0fadabb71-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f65805ca-7829-4bc4-afa9-9cd0fadabb71" (UID: "f65805ca-7829-4bc4-afa9-9cd0fadabb71"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:32:11.565001 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.564870 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95108d71-2908-4a98-8a27-9b2ba48b2e2c-kube-api-access-b7h47" (OuterVolumeSpecName: "kube-api-access-b7h47") pod "95108d71-2908-4a98-8a27-9b2ba48b2e2c" (UID: "95108d71-2908-4a98-8a27-9b2ba48b2e2c"). InnerVolumeSpecName "kube-api-access-b7h47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:11.663556 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.663464 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftsb6\" (UniqueName: \"kubernetes.io/projected/f65805ca-7829-4bc4-afa9-9cd0fadabb71-kube-api-access-ftsb6\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:32:11.663556 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.663494 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7h47\" (UniqueName: \"kubernetes.io/projected/95108d71-2908-4a98-8a27-9b2ba48b2e2c-kube-api-access-b7h47\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:32:11.663556 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:11.663506 2555 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f65805ca-7829-4bc4-afa9-9cd0fadabb71-tls-cert\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:32:12.445523 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.445492 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9mzsc" Jul 02 23:32:12.445523 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.445519 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-868f966f6-krb7k" event={"ID":"df119d6a-d8ca-4a50-a37d-366942fdc1c8","Type":"ContainerStarted","Data":"2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae"} Jul 02 23:32:12.446057 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.445557 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-868f966f6-krb7k" event={"ID":"df119d6a-d8ca-4a50-a37d-366942fdc1c8","Type":"ContainerStarted","Data":"dd957e9d22c3322fd2e264b65d06b217fd4745463845410f094bae89abc773f7"} Jul 02 23:32:12.446057 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.445654 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-whvns" Jul 02 23:32:12.459951 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.459874 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-868f966f6-krb7k" podStartSLOduration=1.031568997 podStartE2EDuration="1.459852579s" totalImagesPullingTime="428.283582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:32:12.459048461 +0000 UTC m=+579.198340919" watchObservedRunningTime="2026-07-02 23:32:12.459852579 +0000 UTC m=+579.199145036" Jul 02 23:32:12.484109 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.484063 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-9mzsc"] Jul 02 23:32:12.487695 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.487652 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pcqzr"] Jul 02 23:32:12.487877 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.487843 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-pcqzr" podUID="42474a58-6ccb-4a38-8cd3-ec6cdbd52954" containerName="authorino" containerID="cri-o://bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f" gracePeriod=30 Jul 02 23:32:12.489534 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.489510 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-9mzsc"] Jul 02 23:32:12.513261 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.513221 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whvns"] Jul 02 23:32:12.518621 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.518591 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-whvns"] Jul 02 23:32:12.781233 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.781196 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pcqzr" Jul 02 23:32:12.875457 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.875422 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwcxk\" (UniqueName: \"kubernetes.io/projected/42474a58-6ccb-4a38-8cd3-ec6cdbd52954-kube-api-access-xwcxk\") pod \"42474a58-6ccb-4a38-8cd3-ec6cdbd52954\" (UID: \"42474a58-6ccb-4a38-8cd3-ec6cdbd52954\") " Jul 02 23:32:12.877927 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.877892 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42474a58-6ccb-4a38-8cd3-ec6cdbd52954-kube-api-access-xwcxk" (OuterVolumeSpecName: "kube-api-access-xwcxk") pod "42474a58-6ccb-4a38-8cd3-ec6cdbd52954" (UID: "42474a58-6ccb-4a38-8cd3-ec6cdbd52954"). InnerVolumeSpecName "kube-api-access-xwcxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:32:12.976354 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:12.976313 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwcxk\" (UniqueName: \"kubernetes.io/projected/42474a58-6ccb-4a38-8cd3-ec6cdbd52954-kube-api-access-xwcxk\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:32:13.451570 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.451539 2555 generic.go:356] "Generic (PLEG): container finished" podID="42474a58-6ccb-4a38-8cd3-ec6cdbd52954" containerID="bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f" exitCode=0 Jul 02 23:32:13.452058 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.451584 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pcqzr" Jul 02 23:32:13.452058 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.451638 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pcqzr" event={"ID":"42474a58-6ccb-4a38-8cd3-ec6cdbd52954","Type":"ContainerDied","Data":"bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f"} Jul 02 23:32:13.452058 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.451678 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pcqzr" event={"ID":"42474a58-6ccb-4a38-8cd3-ec6cdbd52954","Type":"ContainerDied","Data":"7ea934bc334b84b49112dfe9d28902ae9aebb7431ddcf2761417890dc70d8491"} Jul 02 23:32:13.452058 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.451695 2555 scope.go:117] "RemoveContainer" containerID="bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f" Jul 02 23:32:13.460869 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.460851 2555 scope.go:117] "RemoveContainer" containerID="bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f" Jul 02 23:32:13.461190 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:32:13.461168 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f\": container with ID starting with bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f not found: ID does not exist" containerID="bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f" Jul 02 23:32:13.461256 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.461205 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f"} err="failed to get container status \"bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f\": rpc error: code = NotFound desc = could not find container \"bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f\": container with ID starting with bfe615de840739f5844d68459449aeeee526d92829ccdb22d64b8b50f6fc3a5f not found: ID does not exist" Jul 02 23:32:13.471048 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.471006 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pcqzr"] Jul 02 23:32:13.472921 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.472888 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pcqzr"] Jul 02 23:32:13.682995 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.682957 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42474a58-6ccb-4a38-8cd3-ec6cdbd52954" path="/var/lib/kubelet/pods/42474a58-6ccb-4a38-8cd3-ec6cdbd52954/volumes" Jul 02 23:32:13.683267 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.683254 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95108d71-2908-4a98-8a27-9b2ba48b2e2c" path="/var/lib/kubelet/pods/95108d71-2908-4a98-8a27-9b2ba48b2e2c/volumes" Jul 02 23:32:13.683518 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:13.683504 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f65805ca-7829-4bc4-afa9-9cd0fadabb71" path="/var/lib/kubelet/pods/f65805ca-7829-4bc4-afa9-9cd0fadabb71/volumes" Jul 02 23:32:33.598426 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:33.598385 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:32:33.599199 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:33.599177 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:32:53.511842 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:32:53.511787 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:33:21.252755 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:21.252710 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:33:27.137610 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:27.137562 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:33:33.766394 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:33.766340 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:33:48.741987 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:48.741950 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:33:53.273470 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.273427 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk"] Jul 02 23:33:53.273973 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.273833 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42474a58-6ccb-4a38-8cd3-ec6cdbd52954" containerName="authorino" Jul 02 23:33:53.273973 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.273848 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="42474a58-6ccb-4a38-8cd3-ec6cdbd52954" containerName="authorino" Jul 02 23:33:53.273973 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.273925 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="42474a58-6ccb-4a38-8cd3-ec6cdbd52954" containerName="authorino" Jul 02 23:33:53.277165 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.277147 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.280388 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.280346 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 23:33:53.280388 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.280380 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 02 23:33:53.280604 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.280406 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 23:33:53.280604 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.280390 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k5pld\"" Jul 02 23:33:53.286271 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.286248 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk"] Jul 02 23:33:53.417295 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.417254 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.417295 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.417296 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76b9w\" (UniqueName: \"kubernetes.io/projected/42e3d0db-1714-43f5-b280-54378eb86769-kube-api-access-76b9w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.417559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.417392 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.417559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.417454 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e3d0db-1714-43f5-b280-54378eb86769-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.417559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.417481 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.417559 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.417497 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.518357 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518315 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e3d0db-1714-43f5-b280-54378eb86769-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.518578 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518388 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.518578 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518408 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.518578 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518453 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.518578 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518472 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76b9w\" (UniqueName: \"kubernetes.io/projected/42e3d0db-1714-43f5-b280-54378eb86769-kube-api-access-76b9w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.518769 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518597 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.518892 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518872 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.518959 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518919 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.519001 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.518952 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.520912 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.520892 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42e3d0db-1714-43f5-b280-54378eb86769-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.521206 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.521188 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e3d0db-1714-43f5-b280-54378eb86769-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.526734 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.526666 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76b9w\" (UniqueName: \"kubernetes.io/projected/42e3d0db-1714-43f5-b280-54378eb86769-kube-api-access-76b9w\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dwffk\" (UID: \"42e3d0db-1714-43f5-b280-54378eb86769\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.588483 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.588443 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:33:53.739602 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.739574 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk"] Jul 02 23:33:53.741820 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:33:53.741787 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e3d0db_1714_43f5_b280_54378eb86769.slice/crio-469b477fd2c04196337f9b7b4bc34a806e0a22354b51fb60bfec71beba9f2b2c WatchSource:0}: Error finding container 469b477fd2c04196337f9b7b4bc34a806e0a22354b51fb60bfec71beba9f2b2c: Status 404 returned error can't find the container with id 469b477fd2c04196337f9b7b4bc34a806e0a22354b51fb60bfec71beba9f2b2c Jul 02 23:33:53.743082 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.743062 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:33:53.952154 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:53.952099 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" event={"ID":"42e3d0db-1714-43f5-b280-54378eb86769","Type":"ContainerStarted","Data":"469b477fd2c04196337f9b7b4bc34a806e0a22354b51fb60bfec71beba9f2b2c"} Jul 02 23:33:54.958837 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:54.958789 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" event={"ID":"42e3d0db-1714-43f5-b280-54378eb86769","Type":"ContainerStarted","Data":"17e4c1d507d8c8c3d21ac7664ea15c84ff5774c798603e18dc91f4ff9e237eee"} Jul 02 23:33:54.974592 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:54.974535 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" podStartSLOduration=0.959781266 podStartE2EDuration="1.974516005s" totalImagesPullingTime="1.014734739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:33:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:33:54.973414397 +0000 UTC m=+681.712706860" watchObservedRunningTime="2026-07-02 23:33:54.974516005 +0000 UTC m=+681.713808507" Jul 02 23:33:55.140604 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:33:55.140564 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:34:03.589590 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:03.589529 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:34:03.590076 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:03.589622 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:34:03.602331 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:03.602300 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:34:04.015242 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:04.015151 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dwffk" Jul 02 23:34:12.051215 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.051180 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457"] Jul 02 23:34:12.055512 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.055488 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.057928 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.057905 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 02 23:34:12.064304 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.063843 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457"] Jul 02 23:34:12.098583 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.098542 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.098752 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.098598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.098752 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.098631 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.098752 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.098676 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.098855 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.098802 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.098855 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.098830 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtdkr\" (UniqueName: \"kubernetes.io/projected/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-kube-api-access-xtdkr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.199898 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.199861 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.199898 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.199901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.200123 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.199928 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.200123 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.200049 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.200284 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.200146 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.200284 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.200174 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtdkr\" (UniqueName: \"kubernetes.io/projected/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-kube-api-access-xtdkr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.200425 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.200305 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.200425 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.200342 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.200531 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.200504 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.202447 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.202424 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.202704 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.202683 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.207752 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.207726 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtdkr\" (UniqueName: \"kubernetes.io/projected/4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88-kube-api-access-xtdkr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457\" (UID: \"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.368006 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.367915 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:12.502937 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:12.502901 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457"] Jul 02 23:34:12.506000 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:34:12.505969 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e6c4ed3_6fa7_4ae1_b5ee_85635da25a88.slice/crio-618917dfad403d9e9733bbf0ad912607d15e986a9951f9fb20bba48db108c00a WatchSource:0}: Error finding container 618917dfad403d9e9733bbf0ad912607d15e986a9951f9fb20bba48db108c00a: Status 404 returned error can't find the container with id 618917dfad403d9e9733bbf0ad912607d15e986a9951f9fb20bba48db108c00a Jul 02 23:34:13.048582 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:13.048549 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" event={"ID":"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88","Type":"ContainerStarted","Data":"96f8b91b37bd8d452c24f801ee81402a7a15d19f952cd07c3d69e5474dbf4a33"} Jul 02 23:34:13.048774 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:13.048602 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" event={"ID":"4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88","Type":"ContainerStarted","Data":"618917dfad403d9e9733bbf0ad912607d15e986a9951f9fb20bba48db108c00a"} Jul 02 23:34:13.065478 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:13.065424 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" podStartSLOduration=0.888222961 podStartE2EDuration="1.065407573s" totalImagesPullingTime="177.184612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:34:13.063428943 +0000 UTC m=+699.802721418" watchObservedRunningTime="2026-07-02 23:34:13.065407573 +0000 UTC m=+699.804700030" Jul 02 23:34:14.335888 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:14.335854 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:34:22.368778 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:22.368739 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:22.369255 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:22.368801 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:22.381186 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:22.381154 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:34:23.108824 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:34:23.108792 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457" Jul 02 23:35:59.849613 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:35:59.849521 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-868f966f6-krb7k"] Jul 02 23:35:59.850122 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:35:59.849763 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-868f966f6-krb7k" podUID="df119d6a-d8ca-4a50-a37d-366942fdc1c8" containerName="authorino" containerID="cri-o://2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae" gracePeriod=30 Jul 02 23:36:00.105277 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.105206 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:36:00.186862 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.186819 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gq5gz\" (UniqueName: \"kubernetes.io/projected/df119d6a-d8ca-4a50-a37d-366942fdc1c8-kube-api-access-gq5gz\") pod \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\" (UID: \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\") " Jul 02 23:36:00.187061 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.186900 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df119d6a-d8ca-4a50-a37d-366942fdc1c8-tls-cert\") pod \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\" (UID: \"df119d6a-d8ca-4a50-a37d-366942fdc1c8\") " Jul 02 23:36:00.189027 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.188996 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df119d6a-d8ca-4a50-a37d-366942fdc1c8-kube-api-access-gq5gz" (OuterVolumeSpecName: "kube-api-access-gq5gz") pod "df119d6a-d8ca-4a50-a37d-366942fdc1c8" (UID: "df119d6a-d8ca-4a50-a37d-366942fdc1c8"). InnerVolumeSpecName "kube-api-access-gq5gz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:36:00.196873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.196845 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df119d6a-d8ca-4a50-a37d-366942fdc1c8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "df119d6a-d8ca-4a50-a37d-366942fdc1c8" (UID: "df119d6a-d8ca-4a50-a37d-366942fdc1c8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:36:00.288075 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.288021 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gq5gz\" (UniqueName: \"kubernetes.io/projected/df119d6a-d8ca-4a50-a37d-366942fdc1c8-kube-api-access-gq5gz\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:36:00.288075 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.288068 2555 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df119d6a-d8ca-4a50-a37d-366942fdc1c8-tls-cert\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:36:00.561921 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.561888 2555 generic.go:356] "Generic (PLEG): container finished" podID="df119d6a-d8ca-4a50-a37d-366942fdc1c8" containerID="2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae" exitCode=0 Jul 02 23:36:00.562143 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.561939 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-868f966f6-krb7k" Jul 02 23:36:00.562143 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.561975 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-868f966f6-krb7k" event={"ID":"df119d6a-d8ca-4a50-a37d-366942fdc1c8","Type":"ContainerDied","Data":"2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae"} Jul 02 23:36:00.562143 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.562012 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-868f966f6-krb7k" event={"ID":"df119d6a-d8ca-4a50-a37d-366942fdc1c8","Type":"ContainerDied","Data":"dd957e9d22c3322fd2e264b65d06b217fd4745463845410f094bae89abc773f7"} Jul 02 23:36:00.562143 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.562030 2555 scope.go:117] "RemoveContainer" containerID="2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae" Jul 02 23:36:00.570776 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.570752 2555 scope.go:117] "RemoveContainer" containerID="2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae" Jul 02 23:36:00.571042 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:36:00.571023 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae\": container with ID starting with 2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae not found: ID does not exist" containerID="2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae" Jul 02 23:36:00.571114 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.571049 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae"} err="failed to get container status \"2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae\": rpc error: code = NotFound desc = could not find container \"2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae\": container with ID starting with 2941f37083c5e0180c740af956aea3ca84d572428400bc0cd06823693d1d09ae not found: ID does not exist" Jul 02 23:36:00.598280 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.598244 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-868f966f6-krb7k"] Jul 02 23:36:00.604684 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:00.604659 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-868f966f6-krb7k"] Jul 02 23:36:01.682894 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:01.682858 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df119d6a-d8ca-4a50-a37d-366942fdc1c8" path="/var/lib/kubelet/pods/df119d6a-d8ca-4a50-a37d-366942fdc1c8/volumes" Jul 02 23:36:29.236252 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:29.236211 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:36:38.833570 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:38.833533 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:36:49.952128 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:49.952092 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:36:59.735535 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:36:59.735491 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:37:10.241124 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:37:10.241088 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:37:33.627393 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:37:33.627283 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:37:33.628351 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:37:33.628330 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:38:18.144078 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:38:18.144039 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:38:27.535269 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:38:27.535216 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:38:35.643741 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:38:35.643704 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:38:46.434848 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:38:46.434809 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:38:56.137419 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:38:56.137340 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:39:04.737428 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:39:04.737388 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:39:16.735817 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:39:16.735778 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:39:26.033781 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:39:26.033738 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:39:49.443382 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:39:49.443325 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:39:57.936136 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:39:57.936097 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:40:21.436147 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:40:21.436108 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:40:31.332909 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:40:31.332828 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:40:38.044890 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:40:38.044839 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:40:45.736410 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:40:45.736372 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:40:59.545021 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:40:59.544979 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:41:08.442605 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:41:08.442574 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:41:18.837801 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:41:18.837768 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:41:27.432953 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:41:27.432920 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:42:33.657622 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:42:33.657589 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:42:33.660135 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:42:33.660112 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:43:17.445086 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:43:17.445052 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:43:26.340758 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:43:26.340658 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:43:40.736998 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:43:40.736952 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:43:50.444584 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:43:50.444542 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:44:01.637112 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:44:01.637067 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:44:11.837316 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:44:11.837276 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:44:20.940382 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:44:20.940329 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:44:31.635827 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:44:31.635791 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:44:41.837721 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:44:41.837676 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:44:51.643287 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:44:51.643250 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:45:01.437087 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:45:01.436994 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:45:26.537337 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:45:26.537298 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:45:33.341758 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:45:33.341717 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:45:43.445975 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:45:43.445940 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:45:55.132730 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:45:55.132694 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:46:12.237137 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:46:12.237099 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:46:21.639676 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:46:21.639639 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:46:29.242839 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:46:29.242757 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:46:38.234005 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:46:38.233967 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:46:51.146197 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:46:51.146158 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:46:59.728917 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:46:59.728884 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:47:13.540523 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:47:13.540489 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:47:22.436849 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:47:22.436810 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:47:33.436857 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:47:33.436820 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:47:33.687625 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:47:33.687543 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:47:33.693983 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:47:33.693959 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:47:45.229525 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:47:45.229485 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:47:55.436997 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:47:55.436908 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:48:05.036127 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:48:05.036095 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:48:15.943177 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:48:15.943132 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:48:23.935718 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:48:23.935675 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:48:34.237308 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:48:34.237269 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:48:43.435303 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:48:43.435266 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:48:51.738605 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:48:51.738567 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:49:01.233222 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:49:01.233181 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:49:09.339381 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:49:09.339340 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:49:18.539327 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:49:18.539284 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:51:36.143796 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:51:36.143762 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:51:44.738423 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:51:44.738385 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:51:58.232670 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:51:58.232631 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:52:05.841780 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:52:05.841739 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:52:33.718573 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:52:33.718540 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:52:33.723610 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:52:33.723587 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:52:37.631208 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:52:37.631160 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:55:44.838755 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:55:44.838720 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:55:56.542300 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:55:56.542264 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:56:05.542935 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:56:05.542897 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:56:52.241282 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:56:52.241248 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:57:02.538662 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:02.538571 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:57:12.671692 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.671661 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr"] Jul 02 23:57:12.672115 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.672009 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df119d6a-d8ca-4a50-a37d-366942fdc1c8" containerName="authorino" Jul 02 23:57:12.672115 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.672020 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="df119d6a-d8ca-4a50-a37d-366942fdc1c8" containerName="authorino" Jul 02 23:57:12.672115 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.672079 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="df119d6a-d8ca-4a50-a37d-366942fdc1c8" containerName="authorino" Jul 02 23:57:12.675037 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.675020 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.678449 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.678424 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2b5047\"/\"kube-root-ca.crt\"" Jul 02 23:57:12.678575 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.678453 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2b5047\"/\"default-dockercfg-qqgwg\"" Jul 02 23:57:12.678575 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.678424 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2b5047\"/\"openshift-service-ca.crt\"" Jul 02 23:57:12.678575 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.678424 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2b5047\"/\"test-model-2b5047-kserve-self-signed-certs\"" Jul 02 23:57:12.682539 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.682512 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr"] Jul 02 23:57:12.786606 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.786571 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6946c637-53f4-4ffc-94d9-19dca21693d5-tls-certs\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.786797 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.786615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-tmp-dir\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.786797 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.786724 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-home\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.786797 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.786776 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-dshm\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.786945 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.786854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdn8z\" (UniqueName: \"kubernetes.io/projected/6946c637-53f4-4ffc-94d9-19dca21693d5-kube-api-access-jdn8z\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.786998 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.786958 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-model-cache\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.887609 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.887565 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6946c637-53f4-4ffc-94d9-19dca21693d5-tls-certs\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.887609 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.887613 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-tmp-dir\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.887873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.887652 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-home\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.887873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.887675 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-dshm\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.887873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.887707 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdn8z\" (UniqueName: \"kubernetes.io/projected/6946c637-53f4-4ffc-94d9-19dca21693d5-kube-api-access-jdn8z\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.887873 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.887770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-model-cache\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.888129 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.888100 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-tmp-dir\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.888201 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.888144 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-home\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.888201 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.888183 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-model-cache\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.890059 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.890030 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-dshm\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.890397 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.890377 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6946c637-53f4-4ffc-94d9-19dca21693d5-tls-certs\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.894959 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.894938 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdn8z\" (UniqueName: \"kubernetes.io/projected/6946c637-53f4-4ffc-94d9-19dca21693d5-kube-api-access-jdn8z\") pod \"test-model-2b5047-kserve-68747b8749-42ckr\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:12.986085 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:12.986002 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:13.129064 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:13.129038 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr"] Jul 02 23:57:13.131301 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:57:13.131260 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6946c637_53f4_4ffc_94d9_19dca21693d5.slice/crio-dc38421b000df74724933f5d8b391caee5ad8cd0238e0ffa47404872386d10e3 WatchSource:0}: Error finding container dc38421b000df74724933f5d8b391caee5ad8cd0238e0ffa47404872386d10e3: Status 404 returned error can't find the container with id dc38421b000df74724933f5d8b391caee5ad8cd0238e0ffa47404872386d10e3 Jul 02 23:57:13.642983 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:13.642946 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" event={"ID":"6946c637-53f4-4ffc-94d9-19dca21693d5","Type":"ContainerStarted","Data":"689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d"} Jul 02 23:57:13.642983 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:13.642989 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" event={"ID":"6946c637-53f4-4ffc-94d9-19dca21693d5","Type":"ContainerStarted","Data":"dc38421b000df74724933f5d8b391caee5ad8cd0238e0ffa47404872386d10e3"} Jul 02 23:57:13.658430 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:13.658382 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" podStartSLOduration=1.65834759 podStartE2EDuration="1.65834759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:57:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:57:13.657485711 +0000 UTC m=+2080.396778173" watchObservedRunningTime="2026-07-02 23:57:13.65834759 +0000 UTC m=+2080.397640048" Jul 02 23:57:16.859529 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:16.859487 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:57:22.986484 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:22.986450 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:22.987023 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:22.986515 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:22.999404 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:22.999355 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:23.701557 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:23.701528 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:57:33.759274 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:33.759232 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:57:33.765430 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:33.765405 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 02 23:57:42.475987 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.475946 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl"] Jul 02 23:57:42.479416 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.479396 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.481826 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.481707 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6edf51\"/\"kube-root-ca.crt\"" Jul 02 23:57:42.482657 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.482633 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6edf51\"/\"default-dockercfg-zcz84\"" Jul 02 23:57:42.482758 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.482667 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6edf51\"/\"test-model-6edf51-kserve-self-signed-certs\"" Jul 02 23:57:42.482758 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.482710 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6edf51\"/\"openshift-service-ca.crt\"" Jul 02 23:57:42.487321 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.487296 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl"] Jul 02 23:57:42.563771 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.563734 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-model-cache\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.563950 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.563784 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1365ad58-520e-4f72-a399-25aa8dc5317d-tls-certs\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.563950 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.563833 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chjc9\" (UniqueName: \"kubernetes.io/projected/1365ad58-520e-4f72-a399-25aa8dc5317d-kube-api-access-chjc9\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.563950 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.563866 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-tmp-dir\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.563950 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.563921 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-home\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.563950 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.563947 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-dshm\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665153 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665105 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-tmp-dir\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665153 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665165 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-home\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665420 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-dshm\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665420 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665344 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-model-cache\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665535 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1365ad58-520e-4f72-a399-25aa8dc5317d-tls-certs\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665535 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665485 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chjc9\" (UniqueName: \"kubernetes.io/projected/1365ad58-520e-4f72-a399-25aa8dc5317d-kube-api-access-chjc9\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665535 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665511 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-home\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665681 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665551 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-tmp-dir\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.665818 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.665799 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-model-cache\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.667779 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.667759 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-dshm\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.668026 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.668009 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1365ad58-520e-4f72-a399-25aa8dc5317d-tls-certs\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.673115 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.673091 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chjc9\" (UniqueName: \"kubernetes.io/projected/1365ad58-520e-4f72-a399-25aa8dc5317d-kube-api-access-chjc9\") pod \"test-model-6edf51-kserve-5bcf85b6c5-pw5wl\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.789819 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.789732 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:42.925952 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:42.925928 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl"] Jul 02 23:57:42.928533 ip-10-0-137-14 kubenswrapper[2555]: W0702 23:57:42.928508 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1365ad58_520e_4f72_a399_25aa8dc5317d.slice/crio-09e2ef7a0d97fcc7d8b8ccc63665a45596899e4ccc25f2088f40ad6bfb2c08d9 WatchSource:0}: Error finding container 09e2ef7a0d97fcc7d8b8ccc63665a45596899e4ccc25f2088f40ad6bfb2c08d9: Status 404 returned error can't find the container with id 09e2ef7a0d97fcc7d8b8ccc63665a45596899e4ccc25f2088f40ad6bfb2c08d9 Jul 02 23:57:43.791073 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:43.791034 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" event={"ID":"1365ad58-520e-4f72-a399-25aa8dc5317d","Type":"ContainerStarted","Data":"26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd"} Jul 02 23:57:43.791504 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:43.791082 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" event={"ID":"1365ad58-520e-4f72-a399-25aa8dc5317d","Type":"ContainerStarted","Data":"09e2ef7a0d97fcc7d8b8ccc63665a45596899e4ccc25f2088f40ad6bfb2c08d9"} Jul 02 23:57:43.806149 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:43.806099 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" podStartSLOduration=1.806084508 podStartE2EDuration="1.806084508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:57:43.80530589 +0000 UTC m=+2110.544598352" watchObservedRunningTime="2026-07-02 23:57:43.806084508 +0000 UTC m=+2110.545376965" Jul 02 23:57:46.338124 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:46.338084 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:57:52.790732 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:52.790690 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:52.791168 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:52.790958 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:52.803471 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:52.803442 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:52.847743 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:52.847711 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:57:59.637987 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:57:59.637951 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:58:01.562280 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:01.562245 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr"] Jul 02 23:58:02.422947 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:02.422916 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl"] Jul 02 23:58:15.232660 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:15.232618 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:58:16.584962 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.584921 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" podUID="6946c637-53f4-4ffc-94d9-19dca21693d5" containerName="main" containerID="cri-o://689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d" gracePeriod=45 Jul 02 23:58:16.836725 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.836663 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:58:16.950154 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.950115 2555 generic.go:356] "Generic (PLEG): container finished" podID="6946c637-53f4-4ffc-94d9-19dca21693d5" containerID="689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d" exitCode=0 Jul 02 23:58:16.950342 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.950182 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" Jul 02 23:58:16.950342 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.950201 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" event={"ID":"6946c637-53f4-4ffc-94d9-19dca21693d5","Type":"ContainerDied","Data":"689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d"} Jul 02 23:58:16.950342 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.950242 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr" event={"ID":"6946c637-53f4-4ffc-94d9-19dca21693d5","Type":"ContainerDied","Data":"dc38421b000df74724933f5d8b391caee5ad8cd0238e0ffa47404872386d10e3"} Jul 02 23:58:16.950342 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.950258 2555 scope.go:117] "RemoveContainer" containerID="689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d" Jul 02 23:58:16.958893 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.958873 2555 scope.go:117] "RemoveContainer" containerID="689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d" Jul 02 23:58:16.959161 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:58:16.959142 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d\": container with ID starting with 689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d not found: ID does not exist" containerID="689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d" Jul 02 23:58:16.959205 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:16.959170 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d"} err="failed to get container status \"689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d\": rpc error: code = NotFound desc = could not find container \"689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d\": container with ID starting with 689196b6e088de207d8341ab01da22921f97e25eeb8aaa3e56ea6187308f265d not found: ID does not exist" Jul 02 23:58:17.003221 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003190 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-tmp-dir\") pod \"6946c637-53f4-4ffc-94d9-19dca21693d5\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " Jul 02 23:58:17.003221 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003223 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-dshm\") pod \"6946c637-53f4-4ffc-94d9-19dca21693d5\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " Jul 02 23:58:17.003404 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003280 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-home\") pod \"6946c637-53f4-4ffc-94d9-19dca21693d5\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " Jul 02 23:58:17.003404 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003356 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6946c637-53f4-4ffc-94d9-19dca21693d5-tls-certs\") pod \"6946c637-53f4-4ffc-94d9-19dca21693d5\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " Jul 02 23:58:17.003516 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003406 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-model-cache\") pod \"6946c637-53f4-4ffc-94d9-19dca21693d5\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " Jul 02 23:58:17.003516 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003424 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdn8z\" (UniqueName: \"kubernetes.io/projected/6946c637-53f4-4ffc-94d9-19dca21693d5-kube-api-access-jdn8z\") pod \"6946c637-53f4-4ffc-94d9-19dca21693d5\" (UID: \"6946c637-53f4-4ffc-94d9-19dca21693d5\") " Jul 02 23:58:17.003619 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003592 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6946c637-53f4-4ffc-94d9-19dca21693d5" (UID: "6946c637-53f4-4ffc-94d9-19dca21693d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:17.003674 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003612 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-home" (OuterVolumeSpecName: "home") pod "6946c637-53f4-4ffc-94d9-19dca21693d5" (UID: "6946c637-53f4-4ffc-94d9-19dca21693d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:17.003732 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003710 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-home\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.003732 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003711 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "6946c637-53f4-4ffc-94d9-19dca21693d5" (UID: "6946c637-53f4-4ffc-94d9-19dca21693d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:17.003732 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.003727 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-tmp-dir\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.005669 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.005639 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-dshm" (OuterVolumeSpecName: "dshm") pod "6946c637-53f4-4ffc-94d9-19dca21693d5" (UID: "6946c637-53f4-4ffc-94d9-19dca21693d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:17.005771 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.005742 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6946c637-53f4-4ffc-94d9-19dca21693d5-kube-api-access-jdn8z" (OuterVolumeSpecName: "kube-api-access-jdn8z") pod "6946c637-53f4-4ffc-94d9-19dca21693d5" (UID: "6946c637-53f4-4ffc-94d9-19dca21693d5"). InnerVolumeSpecName "kube-api-access-jdn8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:58:17.005858 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.005833 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6946c637-53f4-4ffc-94d9-19dca21693d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6946c637-53f4-4ffc-94d9-19dca21693d5" (UID: "6946c637-53f4-4ffc-94d9-19dca21693d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:58:17.104493 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.104394 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6946c637-53f4-4ffc-94d9-19dca21693d5-tls-certs\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.104493 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.104436 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-model-cache\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.104493 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.104446 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdn8z\" (UniqueName: \"kubernetes.io/projected/6946c637-53f4-4ffc-94d9-19dca21693d5-kube-api-access-jdn8z\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.104493 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.104455 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6946c637-53f4-4ffc-94d9-19dca21693d5-dshm\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.270956 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.270925 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr"] Jul 02 23:58:17.272832 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.272810 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-2b5047/test-model-2b5047-kserve-68747b8749-42ckr"] Jul 02 23:58:17.444437 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.444308 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" podUID="1365ad58-520e-4f72-a399-25aa8dc5317d" containerName="main" containerID="cri-o://26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd" gracePeriod=45 Jul 02 23:58:17.683199 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.683172 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6946c637-53f4-4ffc-94d9-19dca21693d5" path="/var/lib/kubelet/pods/6946c637-53f4-4ffc-94d9-19dca21693d5/volumes" Jul 02 23:58:17.695855 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.695799 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:58:17.811032 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.810996 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chjc9\" (UniqueName: \"kubernetes.io/projected/1365ad58-520e-4f72-a399-25aa8dc5317d-kube-api-access-chjc9\") pod \"1365ad58-520e-4f72-a399-25aa8dc5317d\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " Jul 02 23:58:17.811203 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811075 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-tmp-dir\") pod \"1365ad58-520e-4f72-a399-25aa8dc5317d\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " Jul 02 23:58:17.811203 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811126 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-model-cache\") pod \"1365ad58-520e-4f72-a399-25aa8dc5317d\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " Jul 02 23:58:17.811203 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811144 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-dshm\") pod \"1365ad58-520e-4f72-a399-25aa8dc5317d\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " Jul 02 23:58:17.811203 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811173 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-home\") pod \"1365ad58-520e-4f72-a399-25aa8dc5317d\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " Jul 02 23:58:17.811416 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811221 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1365ad58-520e-4f72-a399-25aa8dc5317d-tls-certs\") pod \"1365ad58-520e-4f72-a399-25aa8dc5317d\" (UID: \"1365ad58-520e-4f72-a399-25aa8dc5317d\") " Jul 02 23:58:17.811416 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811344 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1365ad58-520e-4f72-a399-25aa8dc5317d" (UID: "1365ad58-520e-4f72-a399-25aa8dc5317d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:17.811502 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811461 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-model-cache" (OuterVolumeSpecName: "model-cache") pod "1365ad58-520e-4f72-a399-25aa8dc5317d" (UID: "1365ad58-520e-4f72-a399-25aa8dc5317d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:17.811502 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811478 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-home" (OuterVolumeSpecName: "home") pod "1365ad58-520e-4f72-a399-25aa8dc5317d" (UID: "1365ad58-520e-4f72-a399-25aa8dc5317d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:17.811640 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811602 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-tmp-dir\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.811640 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811624 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-model-cache\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.811640 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.811638 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-home\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.813542 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.813518 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-dshm" (OuterVolumeSpecName: "dshm") pod "1365ad58-520e-4f72-a399-25aa8dc5317d" (UID: "1365ad58-520e-4f72-a399-25aa8dc5317d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:17.813648 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.813582 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1365ad58-520e-4f72-a399-25aa8dc5317d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1365ad58-520e-4f72-a399-25aa8dc5317d" (UID: "1365ad58-520e-4f72-a399-25aa8dc5317d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:58:17.813697 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.813677 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1365ad58-520e-4f72-a399-25aa8dc5317d-kube-api-access-chjc9" (OuterVolumeSpecName: "kube-api-access-chjc9") pod "1365ad58-520e-4f72-a399-25aa8dc5317d" (UID: "1365ad58-520e-4f72-a399-25aa8dc5317d"). InnerVolumeSpecName "kube-api-access-chjc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:58:17.913121 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.913082 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1365ad58-520e-4f72-a399-25aa8dc5317d-dshm\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.913121 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.913111 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1365ad58-520e-4f72-a399-25aa8dc5317d-tls-certs\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.913121 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.913122 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chjc9\" (UniqueName: \"kubernetes.io/projected/1365ad58-520e-4f72-a399-25aa8dc5317d-kube-api-access-chjc9\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 02 23:58:17.957451 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.957355 2555 generic.go:356] "Generic (PLEG): container finished" podID="1365ad58-520e-4f72-a399-25aa8dc5317d" containerID="26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd" exitCode=0 Jul 02 23:58:17.957451 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.957420 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" Jul 02 23:58:17.957451 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.957447 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" event={"ID":"1365ad58-520e-4f72-a399-25aa8dc5317d","Type":"ContainerDied","Data":"26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd"} Jul 02 23:58:17.957634 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.957476 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl" event={"ID":"1365ad58-520e-4f72-a399-25aa8dc5317d","Type":"ContainerDied","Data":"09e2ef7a0d97fcc7d8b8ccc63665a45596899e4ccc25f2088f40ad6bfb2c08d9"} Jul 02 23:58:17.957634 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.957492 2555 scope.go:117] "RemoveContainer" containerID="26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd" Jul 02 23:58:17.966396 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.966354 2555 scope.go:117] "RemoveContainer" containerID="26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd" Jul 02 23:58:17.966658 ip-10-0-137-14 kubenswrapper[2555]: E0702 23:58:17.966634 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd\": container with ID starting with 26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd not found: ID does not exist" containerID="26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd" Jul 02 23:58:17.966721 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.966670 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd"} err="failed to get container status \"26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd\": rpc error: code = NotFound desc = could not find container \"26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd\": container with ID starting with 26ae6d8cf77134bca6abffa395d841bb255214444cfd7d818c46883689a3a6cd not found: ID does not exist" Jul 02 23:58:17.977479 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.977448 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl"] Jul 02 23:58:17.978964 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:17.978944 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-6edf51/test-model-6edf51-kserve-5bcf85b6c5-pw5wl"] Jul 02 23:58:19.682972 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:19.682936 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1365ad58-520e-4f72-a399-25aa8dc5317d" path="/var/lib/kubelet/pods/1365ad58-520e-4f72-a399-25aa8dc5317d/volumes" Jul 02 23:58:44.341870 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:44.341826 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:58:58.048039 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:58:58.048004 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:59:15.441686 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:59:15.441650 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:59:25.036107 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:59:25.036063 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:59:38.542043 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:59:38.542008 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 02 23:59:53.740478 ip-10-0-137-14 kubenswrapper[2555]: I0702 23:59:53.740439 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:00:09.329669 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.329584 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz"] Jul 03 00:00:09.330020 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.329950 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1365ad58-520e-4f72-a399-25aa8dc5317d" containerName="main" Jul 03 00:00:09.330020 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.329961 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1365ad58-520e-4f72-a399-25aa8dc5317d" containerName="main" Jul 03 00:00:09.330020 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.329983 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6946c637-53f4-4ffc-94d9-19dca21693d5" containerName="main" Jul 03 00:00:09.330020 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.329988 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6946c637-53f4-4ffc-94d9-19dca21693d5" containerName="main" Jul 03 00:00:09.330147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.330038 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6946c637-53f4-4ffc-94d9-19dca21693d5" containerName="main" Jul 03 00:00:09.330147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.330049 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1365ad58-520e-4f72-a399-25aa8dc5317d" containerName="main" Jul 03 00:00:09.332993 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.332966 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.336559 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.336537 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"default-dockercfg-n8k7w\"" Jul 03 00:00:09.337671 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.337656 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"openshift-service-ca.crt\"" Jul 03 00:00:09.337732 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.337673 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"auth-test-model-c1893b-kserve-self-signed-certs\"" Jul 03 00:00:09.337732 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.337689 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"kube-root-ca.crt\"" Jul 03 00:00:09.353759 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.353728 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz"] Jul 03 00:00:09.399046 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.399009 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tmp-dir\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.399203 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.399072 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jprrc\" (UniqueName: \"kubernetes.io/projected/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-kube-api-access-jprrc\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.399203 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.399126 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-home\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.399203 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.399151 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-dshm\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.399203 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.399181 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-model-cache\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.399380 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.399218 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tls-certs\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500092 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500051 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tmp-dir\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500273 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500106 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jprrc\" (UniqueName: \"kubernetes.io/projected/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-kube-api-access-jprrc\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500273 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500144 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-home\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500273 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500161 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-dshm\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500273 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500181 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-model-cache\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500273 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500239 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tls-certs\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500654 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500625 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tmp-dir\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500723 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500656 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-model-cache\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.500723 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.500696 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-home\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.502779 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.502751 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-dshm\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.502937 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.502921 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tls-certs\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.508175 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.508144 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jprrc\" (UniqueName: \"kubernetes.io/projected/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-kube-api-access-jprrc\") pod \"auth-test-model-c1893b-kserve-696d479987-trfvz\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.659482 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.659387 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:09.802959 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:09.802929 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz"] Jul 03 00:00:09.805240 ip-10-0-137-14 kubenswrapper[2555]: W0703 00:00:09.805213 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc66ed28_1c3f_4643_ad3c_f0f52969d7d4.slice/crio-e3f7c406da8a46fac7298b9be6f28792bc688ac07e346b2294365d6d4261fa92 WatchSource:0}: Error finding container e3f7c406da8a46fac7298b9be6f28792bc688ac07e346b2294365d6d4261fa92: Status 404 returned error can't find the container with id e3f7c406da8a46fac7298b9be6f28792bc688ac07e346b2294365d6d4261fa92 Jul 03 00:00:10.487800 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:10.487756 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" event={"ID":"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4","Type":"ContainerStarted","Data":"5840e433d0d8934fd6aa585ff477c5b199117a1e942d2970e2ffa8c9248a8117"} Jul 03 00:00:10.487800 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:10.487807 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" event={"ID":"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4","Type":"ContainerStarted","Data":"e3f7c406da8a46fac7298b9be6f28792bc688ac07e346b2294365d6d4261fa92"} Jul 03 00:00:10.504119 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:10.504064 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" podStartSLOduration=1.504047277 podStartE2EDuration="1.504047277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:00:10.503333211 +0000 UTC m=+2257.242625670" watchObservedRunningTime="2026-07-03 00:00:10.504047277 +0000 UTC m=+2257.243339729" Jul 03 00:00:12.337040 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:12.336997 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:00:19.659817 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:19.659771 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:19.660206 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:19.659835 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:19.672242 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:19.672213 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:20.546874 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:20.546845 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:00:55.372464 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.372427 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v"] Jul 03 00:00:55.375957 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.375934 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.378287 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.378265 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"kube-root-ca.crt\"" Jul 03 00:00:55.379334 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.379317 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"default-dockercfg-r9b5t\"" Jul 03 00:00:55.379430 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.379349 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"auth-test-model-c04a64-kserve-self-signed-certs\"" Jul 03 00:00:55.379430 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.379319 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"openshift-service-ca.crt\"" Jul 03 00:00:55.383401 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.383353 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v"] Jul 03 00:00:55.519324 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.519282 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e61966e-8a95-4372-8742-25b336994da7-tls-certs\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.519324 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.519332 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-model-cache\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.519562 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.519403 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh7px\" (UniqueName: \"kubernetes.io/projected/6e61966e-8a95-4372-8742-25b336994da7-kube-api-access-vh7px\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.519562 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.519427 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-home\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.519562 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.519466 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-dshm\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.519562 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.519497 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-tmp-dir\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.620638 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.620601 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh7px\" (UniqueName: \"kubernetes.io/projected/6e61966e-8a95-4372-8742-25b336994da7-kube-api-access-vh7px\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.620832 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.620648 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-home\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.620832 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.620708 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-dshm\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.620832 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.620739 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-tmp-dir\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.620832 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.620799 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e61966e-8a95-4372-8742-25b336994da7-tls-certs\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.621084 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.620839 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-model-cache\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.621146 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.621077 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-home\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.621204 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.621180 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-tmp-dir\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.621259 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.621229 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-model-cache\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.623747 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.623686 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-dshm\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.623888 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.623872 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e61966e-8a95-4372-8742-25b336994da7-tls-certs\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.627598 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.627575 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh7px\" (UniqueName: \"kubernetes.io/projected/6e61966e-8a95-4372-8742-25b336994da7-kube-api-access-vh7px\") pod \"auth-test-model-c04a64-kserve-d898864cf-mfq7v\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.686321 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.686286 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:00:55.819418 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:55.819391 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v"] Jul 03 00:00:55.823726 ip-10-0-137-14 kubenswrapper[2555]: W0703 00:00:55.823684 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e61966e_8a95_4372_8742_25b336994da7.slice/crio-7bd9cb7709040766928a4a0b6ac55db9d794c9374d7498efe2005b2e8602519c WatchSource:0}: Error finding container 7bd9cb7709040766928a4a0b6ac55db9d794c9374d7498efe2005b2e8602519c: Status 404 returned error can't find the container with id 7bd9cb7709040766928a4a0b6ac55db9d794c9374d7498efe2005b2e8602519c Jul 03 00:00:56.707250 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:56.707205 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" event={"ID":"6e61966e-8a95-4372-8742-25b336994da7","Type":"ContainerStarted","Data":"cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d"} Jul 03 00:00:56.707250 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:56.707253 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" event={"ID":"6e61966e-8a95-4372-8742-25b336994da7","Type":"ContainerStarted","Data":"7bd9cb7709040766928a4a0b6ac55db9d794c9374d7498efe2005b2e8602519c"} Jul 03 00:00:56.727827 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:56.727768 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" podStartSLOduration=1.727751661 podStartE2EDuration="1.727751661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:00:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:00:56.726636362 +0000 UTC m=+2303.465928818" watchObservedRunningTime="2026-07-03 00:00:56.727751661 +0000 UTC m=+2303.467044117" Jul 03 00:00:59.032073 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:00:59.032037 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:01:05.687003 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:01:05.686961 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:01:05.687502 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:01:05.687101 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:01:05.699207 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:01:05.699174 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:01:05.761853 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:01:05.761818 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:01:14.241901 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:01:14.241869 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:01:28.830605 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:01:28.830515 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:01:44.637564 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:01:44.637519 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:01:57.145957 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:01:57.145916 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:02:15.234261 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:02:15.234220 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:02:32.347565 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:02:32.347526 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:02:33.788568 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:02:33.788537 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 03 00:02:33.796238 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:02:33.796216 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8mhj9_c4086f1f-8cb6-4adc-9364-62e754a6259a/ovn-acl-logging/0.log" Jul 03 00:02:46.116176 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:02:46.116129 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz"] Jul 03 00:02:47.180588 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:02:47.180548 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v"] Jul 03 00:02:49.242789 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:02:49.242752 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:03:01.137047 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.136982 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" podUID="dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" containerName="main" containerID="cri-o://5840e433d0d8934fd6aa585ff477c5b199117a1e942d2970e2ffa8c9248a8117" gracePeriod=45 Jul 03 00:03:01.303647 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.303614 2555 generic.go:356] "Generic (PLEG): container finished" podID="dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" containerID="5840e433d0d8934fd6aa585ff477c5b199117a1e942d2970e2ffa8c9248a8117" exitCode=0 Jul 03 00:03:01.303819 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.303696 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" event={"ID":"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4","Type":"ContainerDied","Data":"5840e433d0d8934fd6aa585ff477c5b199117a1e942d2970e2ffa8c9248a8117"} Jul 03 00:03:01.395036 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.394972 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:03:01.522277 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522226 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tmp-dir\") pod \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " Jul 03 00:03:01.522488 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522289 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tls-certs\") pod \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " Jul 03 00:03:01.522488 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522314 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jprrc\" (UniqueName: \"kubernetes.io/projected/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-kube-api-access-jprrc\") pod \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " Jul 03 00:03:01.522488 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522352 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-home\") pod \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " Jul 03 00:03:01.522488 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522447 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-dshm\") pod \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " Jul 03 00:03:01.522678 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522496 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-model-cache\") pod \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\" (UID: \"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4\") " Jul 03 00:03:01.522678 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522601 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" (UID: "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:03:01.522678 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522657 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-home" (OuterVolumeSpecName: "home") pod "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" (UID: "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:03:01.522863 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522838 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-model-cache" (OuterVolumeSpecName: "model-cache") pod "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" (UID: "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:03:01.522928 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522916 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tmp-dir\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:01.522971 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.522934 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-home\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:01.524800 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.524770 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-dshm" (OuterVolumeSpecName: "dshm") pod "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" (UID: "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:03:01.524912 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.524815 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-kube-api-access-jprrc" (OuterVolumeSpecName: "kube-api-access-jprrc") pod "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" (UID: "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4"). InnerVolumeSpecName "kube-api-access-jprrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:03:01.524912 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.524817 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" (UID: "dc66ed28-1c3f-4643-ad3c-f0f52969d7d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 00:03:01.624148 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.624107 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-dshm\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:01.624148 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.624142 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-model-cache\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:01.624148 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.624154 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-tls-certs\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:01.624437 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:01.624163 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jprrc\" (UniqueName: \"kubernetes.io/projected/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4-kube-api-access-jprrc\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:02.201810 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.201745 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" podUID="6e61966e-8a95-4372-8742-25b336994da7" containerName="main" containerID="cri-o://cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d" gracePeriod=45 Jul 03 00:03:02.317430 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.317394 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" event={"ID":"dc66ed28-1c3f-4643-ad3c-f0f52969d7d4","Type":"ContainerDied","Data":"e3f7c406da8a46fac7298b9be6f28792bc688ac07e346b2294365d6d4261fa92"} Jul 03 00:03:02.317610 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.317444 2555 scope.go:117] "RemoveContainer" containerID="5840e433d0d8934fd6aa585ff477c5b199117a1e942d2970e2ffa8c9248a8117" Jul 03 00:03:02.317610 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.317443 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz" Jul 03 00:03:02.333288 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.333255 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz"] Jul 03 00:03:02.336495 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.336471 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/auth-test-model-c1893b-kserve-696d479987-trfvz"] Jul 03 00:03:02.466785 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.466720 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:03:02.533707 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.533676 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-home\") pod \"6e61966e-8a95-4372-8742-25b336994da7\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " Jul 03 00:03:02.533907 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.533718 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh7px\" (UniqueName: \"kubernetes.io/projected/6e61966e-8a95-4372-8742-25b336994da7-kube-api-access-vh7px\") pod \"6e61966e-8a95-4372-8742-25b336994da7\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " Jul 03 00:03:02.533907 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.533754 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-tmp-dir\") pod \"6e61966e-8a95-4372-8742-25b336994da7\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " Jul 03 00:03:02.533907 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.533808 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-model-cache\") pod \"6e61966e-8a95-4372-8742-25b336994da7\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " Jul 03 00:03:02.533907 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.533865 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e61966e-8a95-4372-8742-25b336994da7-tls-certs\") pod \"6e61966e-8a95-4372-8742-25b336994da7\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " Jul 03 00:03:02.533907 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.533905 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-dshm\") pod \"6e61966e-8a95-4372-8742-25b336994da7\" (UID: \"6e61966e-8a95-4372-8742-25b336994da7\") " Jul 03 00:03:02.534162 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.534016 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-home" (OuterVolumeSpecName: "home") pod "6e61966e-8a95-4372-8742-25b336994da7" (UID: "6e61966e-8a95-4372-8742-25b336994da7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:03:02.534162 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.534070 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6e61966e-8a95-4372-8742-25b336994da7" (UID: "6e61966e-8a95-4372-8742-25b336994da7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:03:02.534233 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.534184 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-model-cache" (OuterVolumeSpecName: "model-cache") pod "6e61966e-8a95-4372-8742-25b336994da7" (UID: "6e61966e-8a95-4372-8742-25b336994da7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:03:02.534291 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.534275 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-home\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:02.534326 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.534311 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-tmp-dir\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:02.534326 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.534320 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-model-cache\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:02.536109 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.536085 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e61966e-8a95-4372-8742-25b336994da7-kube-api-access-vh7px" (OuterVolumeSpecName: "kube-api-access-vh7px") pod "6e61966e-8a95-4372-8742-25b336994da7" (UID: "6e61966e-8a95-4372-8742-25b336994da7"). InnerVolumeSpecName "kube-api-access-vh7px". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:03:02.536233 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.536177 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e61966e-8a95-4372-8742-25b336994da7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6e61966e-8a95-4372-8742-25b336994da7" (UID: "6e61966e-8a95-4372-8742-25b336994da7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 00:03:02.536291 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.536221 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-dshm" (OuterVolumeSpecName: "dshm") pod "6e61966e-8a95-4372-8742-25b336994da7" (UID: "6e61966e-8a95-4372-8742-25b336994da7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:03:02.634944 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.634903 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh7px\" (UniqueName: \"kubernetes.io/projected/6e61966e-8a95-4372-8742-25b336994da7-kube-api-access-vh7px\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:02.634944 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.634939 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e61966e-8a95-4372-8742-25b336994da7-tls-certs\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:02.634944 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:02.634950 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e61966e-8a95-4372-8742-25b336994da7-dshm\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:03:03.325044 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.325007 2555 generic.go:356] "Generic (PLEG): container finished" podID="6e61966e-8a95-4372-8742-25b336994da7" containerID="cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d" exitCode=0 Jul 03 00:03:03.325692 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.325055 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" Jul 03 00:03:03.325692 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.325088 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" event={"ID":"6e61966e-8a95-4372-8742-25b336994da7","Type":"ContainerDied","Data":"cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d"} Jul 03 00:03:03.325692 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.325132 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v" event={"ID":"6e61966e-8a95-4372-8742-25b336994da7","Type":"ContainerDied","Data":"7bd9cb7709040766928a4a0b6ac55db9d794c9374d7498efe2005b2e8602519c"} Jul 03 00:03:03.325692 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.325150 2555 scope.go:117] "RemoveContainer" containerID="cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d" Jul 03 00:03:03.334269 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.334250 2555 scope.go:117] "RemoveContainer" containerID="cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d" Jul 03 00:03:03.334560 ip-10-0-137-14 kubenswrapper[2555]: E0703 00:03:03.334541 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d\": container with ID starting with cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d not found: ID does not exist" containerID="cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d" Jul 03 00:03:03.334613 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.334570 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d"} err="failed to get container status \"cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d\": rpc error: code = NotFound desc = could not find container \"cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d\": container with ID starting with cf555879bd8bfd4a16dd29ddeec447eff3d411ea6b1b71c4e669e7bb477de30d not found: ID does not exist" Jul 03 00:03:03.344940 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.344910 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v"] Jul 03 00:03:03.348645 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.348621 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/auth-test-model-c04a64-kserve-d898864cf-mfq7v"] Jul 03 00:03:03.683333 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.683249 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e61966e-8a95-4372-8742-25b336994da7" path="/var/lib/kubelet/pods/6e61966e-8a95-4372-8742-25b336994da7/volumes" Jul 03 00:03:03.683751 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.683734 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" path="/var/lib/kubelet/pods/dc66ed28-1c3f-4643-ad3c-f0f52969d7d4/volumes" Jul 03 00:03:03.937684 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:03.937594 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:03:22.078926 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.078887 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn"] Jul 03 00:03:22.079345 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.079226 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" containerName="main" Jul 03 00:03:22.079345 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.079247 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" containerName="main" Jul 03 00:03:22.079345 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.079266 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e61966e-8a95-4372-8742-25b336994da7" containerName="main" Jul 03 00:03:22.079345 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.079271 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e61966e-8a95-4372-8742-25b336994da7" containerName="main" Jul 03 00:03:22.079573 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.079372 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e61966e-8a95-4372-8742-25b336994da7" containerName="main" Jul 03 00:03:22.079573 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.079383 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc66ed28-1c3f-4643-ad3c-f0f52969d7d4" containerName="main" Jul 03 00:03:22.082249 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.082232 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.084824 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.084789 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"openshift-service-ca.crt\"" Jul 03 00:03:22.084965 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.084933 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"kube-root-ca.crt\"" Jul 03 00:03:22.085793 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.085772 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"sub-test-model-c04a64-kserve-self-signed-certs\"" Jul 03 00:03:22.085925 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.085818 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"default-dockercfg-r9b5t\"" Jul 03 00:03:22.089618 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.089594 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn"] Jul 03 00:03:22.218037 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.218000 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44fe2545-9056-4e17-9705-9e19aa8b73dc-tls-certs\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.218212 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.218077 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-home\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.218212 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.218101 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-dshm\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.218212 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.218116 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-model-cache\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.218212 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.218136 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k52vf\" (UniqueName: \"kubernetes.io/projected/44fe2545-9056-4e17-9705-9e19aa8b73dc-kube-api-access-k52vf\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.218355 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.218242 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-tmp-dir\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319256 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319208 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44fe2545-9056-4e17-9705-9e19aa8b73dc-tls-certs\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319470 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319288 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-home\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319470 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319309 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-dshm\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319470 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319324 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-model-cache\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319470 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319341 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k52vf\" (UniqueName: \"kubernetes.io/projected/44fe2545-9056-4e17-9705-9e19aa8b73dc-kube-api-access-k52vf\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319678 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319523 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-tmp-dir\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319792 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319772 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-home\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319852 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319838 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-model-cache\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.319950 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.319928 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-tmp-dir\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.321897 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.321871 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-dshm\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.322008 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.321960 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44fe2545-9056-4e17-9705-9e19aa8b73dc-tls-certs\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.328433 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.328413 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k52vf\" (UniqueName: \"kubernetes.io/projected/44fe2545-9056-4e17-9705-9e19aa8b73dc-kube-api-access-k52vf\") pod \"sub-test-model-c04a64-kserve-5cd84cdcff-slxjn\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.393387 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.393278 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:22.531384 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:22.531343 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn"] Jul 03 00:03:22.533925 ip-10-0-137-14 kubenswrapper[2555]: W0703 00:03:22.533892 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44fe2545_9056_4e17_9705_9e19aa8b73dc.slice/crio-44b9854fc64a19279a886bdca955e88d409ea3ec0fe99043f32e81df166096da WatchSource:0}: Error finding container 44b9854fc64a19279a886bdca955e88d409ea3ec0fe99043f32e81df166096da: Status 404 returned error can't find the container with id 44b9854fc64a19279a886bdca955e88d409ea3ec0fe99043f32e81df166096da Jul 03 00:03:23.425181 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:23.425140 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" event={"ID":"44fe2545-9056-4e17-9705-9e19aa8b73dc","Type":"ContainerStarted","Data":"617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2"} Jul 03 00:03:23.425181 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:23.425186 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" event={"ID":"44fe2545-9056-4e17-9705-9e19aa8b73dc","Type":"ContainerStarted","Data":"44b9854fc64a19279a886bdca955e88d409ea3ec0fe99043f32e81df166096da"} Jul 03 00:03:23.439800 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:23.439756 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" podStartSLOduration=1.43973914 podStartE2EDuration="1.43973914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:03:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:03:23.439108595 +0000 UTC m=+2450.178401055" watchObservedRunningTime="2026-07-03 00:03:23.43973914 +0000 UTC m=+2450.179031598" Jul 03 00:03:25.745668 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:25.745630 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:03:32.393600 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:32.393566 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:32.394017 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:32.393612 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:32.406248 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:32.406218 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:03:32.479377 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:03:32.479340 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:04:02.343103 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:02.343066 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:04:19.245399 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:19.245347 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:04:25.278534 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:25.278452 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn"] Jul 03 00:04:31.134350 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.134316 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst"] Jul 03 00:04:31.137723 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.137705 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.141620 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.141594 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"openshift-service-ca.crt\"" Jul 03 00:04:31.141765 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.141618 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"kube-root-ca.crt\"" Jul 03 00:04:31.141765 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.141650 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"default-dockercfg-n8k7w\"" Jul 03 00:04:31.141765 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.141619 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"rate-test-model-c1893b-kserve-self-signed-certs\"" Jul 03 00:04:31.155603 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.155576 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst"] Jul 03 00:04:31.260901 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.260857 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-model-cache\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.261089 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.260917 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tls-certs\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.261089 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.260982 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-home\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.261089 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.261038 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-dshm\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.261214 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.261090 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tmp-dir\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.261214 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.261109 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkwp\" (UniqueName: \"kubernetes.io/projected/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-kube-api-access-vpkwp\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.361885 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.361841 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tls-certs\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.362081 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.361900 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-home\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.362081 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.361935 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-dshm\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.362081 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.361970 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tmp-dir\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.362081 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.361988 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkwp\" (UniqueName: \"kubernetes.io/projected/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-kube-api-access-vpkwp\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.362081 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.362033 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-model-cache\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.362481 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.362456 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-home\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.362560 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.362495 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-model-cache\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.362560 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.362534 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tmp-dir\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.364545 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.364520 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-dshm\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.364785 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.364769 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tls-certs\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.369379 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.369335 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkwp\" (UniqueName: \"kubernetes.io/projected/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-kube-api-access-vpkwp\") pod \"rate-test-model-c1893b-kserve-7f56c44f87-9ljst\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.446933 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.446838 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:31.584085 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.584051 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst"] Jul 03 00:04:31.587190 ip-10-0-137-14 kubenswrapper[2555]: W0703 00:04:31.587157 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd121117_bd04_4d57_9fd7_1c0df52d4c1c.slice/crio-2c20658ea94e0eb6239ac39fc65f11d3e1384ac01d3281bc56237f5b8a531377 WatchSource:0}: Error finding container 2c20658ea94e0eb6239ac39fc65f11d3e1384ac01d3281bc56237f5b8a531377: Status 404 returned error can't find the container with id 2c20658ea94e0eb6239ac39fc65f11d3e1384ac01d3281bc56237f5b8a531377 Jul 03 00:04:31.762136 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.762044 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" event={"ID":"bd121117-bd04-4d57-9fd7-1c0df52d4c1c","Type":"ContainerStarted","Data":"06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e"} Jul 03 00:04:31.762136 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.762086 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" event={"ID":"bd121117-bd04-4d57-9fd7-1c0df52d4c1c","Type":"ContainerStarted","Data":"2c20658ea94e0eb6239ac39fc65f11d3e1384ac01d3281bc56237f5b8a531377"} Jul 03 00:04:31.778298 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:31.778244 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" podStartSLOduration=0.778226869 podStartE2EDuration="778.226869ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:04:31.776769301 +0000 UTC m=+2518.516061764" watchObservedRunningTime="2026-07-03 00:04:31.778226869 +0000 UTC m=+2518.517519345" Jul 03 00:04:35.636352 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:35.636314 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:04:40.299397 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.299322 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" podUID="44fe2545-9056-4e17-9705-9e19aa8b73dc" containerName="main" containerID="cri-o://617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2" gracePeriod=45 Jul 03 00:04:40.557057 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.556991 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:04:40.648581 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648551 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-tmp-dir\") pod \"44fe2545-9056-4e17-9705-9e19aa8b73dc\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " Jul 03 00:04:40.648765 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648631 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-model-cache\") pod \"44fe2545-9056-4e17-9705-9e19aa8b73dc\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " Jul 03 00:04:40.648765 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648651 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-dshm\") pod \"44fe2545-9056-4e17-9705-9e19aa8b73dc\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " Jul 03 00:04:40.648765 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648665 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-home\") pod \"44fe2545-9056-4e17-9705-9e19aa8b73dc\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " Jul 03 00:04:40.648765 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648700 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44fe2545-9056-4e17-9705-9e19aa8b73dc-tls-certs\") pod \"44fe2545-9056-4e17-9705-9e19aa8b73dc\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " Jul 03 00:04:40.648765 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648716 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k52vf\" (UniqueName: \"kubernetes.io/projected/44fe2545-9056-4e17-9705-9e19aa8b73dc-kube-api-access-k52vf\") pod \"44fe2545-9056-4e17-9705-9e19aa8b73dc\" (UID: \"44fe2545-9056-4e17-9705-9e19aa8b73dc\") " Jul 03 00:04:40.649002 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648877 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "44fe2545-9056-4e17-9705-9e19aa8b73dc" (UID: "44fe2545-9056-4e17-9705-9e19aa8b73dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:04:40.649002 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648898 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44fe2545-9056-4e17-9705-9e19aa8b73dc" (UID: "44fe2545-9056-4e17-9705-9e19aa8b73dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:04:40.649002 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.648941 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-home" (OuterVolumeSpecName: "home") pod "44fe2545-9056-4e17-9705-9e19aa8b73dc" (UID: "44fe2545-9056-4e17-9705-9e19aa8b73dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:04:40.649167 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.649024 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-home\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:04:40.649167 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.649037 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-tmp-dir\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:04:40.649167 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.649047 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-model-cache\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:04:40.650910 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.650876 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-dshm" (OuterVolumeSpecName: "dshm") pod "44fe2545-9056-4e17-9705-9e19aa8b73dc" (UID: "44fe2545-9056-4e17-9705-9e19aa8b73dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:04:40.651025 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.650960 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44fe2545-9056-4e17-9705-9e19aa8b73dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44fe2545-9056-4e17-9705-9e19aa8b73dc" (UID: "44fe2545-9056-4e17-9705-9e19aa8b73dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 00:04:40.651025 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.650987 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44fe2545-9056-4e17-9705-9e19aa8b73dc-kube-api-access-k52vf" (OuterVolumeSpecName: "kube-api-access-k52vf") pod "44fe2545-9056-4e17-9705-9e19aa8b73dc" (UID: "44fe2545-9056-4e17-9705-9e19aa8b73dc"). InnerVolumeSpecName "kube-api-access-k52vf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:04:40.750437 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.750356 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44fe2545-9056-4e17-9705-9e19aa8b73dc-dshm\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:04:40.750437 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.750434 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44fe2545-9056-4e17-9705-9e19aa8b73dc-tls-certs\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:04:40.750437 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.750445 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k52vf\" (UniqueName: \"kubernetes.io/projected/44fe2545-9056-4e17-9705-9e19aa8b73dc-kube-api-access-k52vf\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:04:40.811653 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.811546 2555 generic.go:356] "Generic (PLEG): container finished" podID="44fe2545-9056-4e17-9705-9e19aa8b73dc" containerID="617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2" exitCode=0 Jul 03 00:04:40.811653 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.811590 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" Jul 03 00:04:40.811653 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.811629 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" event={"ID":"44fe2545-9056-4e17-9705-9e19aa8b73dc","Type":"ContainerDied","Data":"617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2"} Jul 03 00:04:40.811924 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.811686 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn" event={"ID":"44fe2545-9056-4e17-9705-9e19aa8b73dc","Type":"ContainerDied","Data":"44b9854fc64a19279a886bdca955e88d409ea3ec0fe99043f32e81df166096da"} Jul 03 00:04:40.811924 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.811707 2555 scope.go:117] "RemoveContainer" containerID="617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2" Jul 03 00:04:40.821150 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.821132 2555 scope.go:117] "RemoveContainer" containerID="617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2" Jul 03 00:04:40.821439 ip-10-0-137-14 kubenswrapper[2555]: E0703 00:04:40.821420 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2\": container with ID starting with 617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2 not found: ID does not exist" containerID="617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2" Jul 03 00:04:40.821496 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.821448 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2"} err="failed to get container status \"617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2\": rpc error: code = NotFound desc = could not find container \"617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2\": container with ID starting with 617e10146b64ee5b6b19dd3d0a8b67b9950d828c0284e873512bca97924572e2 not found: ID does not exist" Jul 03 00:04:40.831243 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.831202 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn"] Jul 03 00:04:40.833464 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:40.833437 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/sub-test-model-c04a64-kserve-5cd84cdcff-slxjn"] Jul 03 00:04:41.447975 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:41.447934 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:41.448511 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:41.447999 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:41.460492 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:41.460464 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:41.682917 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:41.682878 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44fe2545-9056-4e17-9705-9e19aa8b73dc" path="/var/lib/kubelet/pods/44fe2545-9056-4e17-9705-9e19aa8b73dc/volumes" Jul 03 00:04:41.828937 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:41.828899 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:04:54.174398 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.174346 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk"] Jul 03 00:04:54.174764 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.174754 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44fe2545-9056-4e17-9705-9e19aa8b73dc" containerName="main" Jul 03 00:04:54.174806 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.174766 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="44fe2545-9056-4e17-9705-9e19aa8b73dc" containerName="main" Jul 03 00:04:54.174858 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.174847 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="44fe2545-9056-4e17-9705-9e19aa8b73dc" containerName="main" Jul 03 00:04:54.179421 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.179399 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.181660 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.181637 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"kube-root-ca.crt\"" Jul 03 00:04:54.181790 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.181665 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"openshift-service-ca.crt\"" Jul 03 00:04:54.182721 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.182681 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"rate-test-model-c04a64-kserve-self-signed-certs\"" Jul 03 00:04:54.182816 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.182746 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c04a64\"/\"default-dockercfg-r9b5t\"" Jul 03 00:04:54.186090 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.186068 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk"] Jul 03 00:04:54.272301 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.272261 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-model-cache\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.272505 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.272309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tmp-dir\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.272505 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.272337 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbt84\" (UniqueName: \"kubernetes.io/projected/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-kube-api-access-hbt84\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.272505 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.272395 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-home\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.272505 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.272411 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-dshm\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.272505 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.272426 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tls-certs\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373079 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373036 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-model-cache\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373079 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373088 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tmp-dir\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373325 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373115 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbt84\" (UniqueName: \"kubernetes.io/projected/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-kube-api-access-hbt84\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373325 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373136 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-home\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373325 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373152 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-dshm\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373325 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373165 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tls-certs\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373565 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373528 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-model-cache\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373624 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373577 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-home\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.373624 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.373598 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tmp-dir\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.375699 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.375674 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-dshm\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.375841 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.375824 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tls-certs\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.380587 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.380561 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbt84\" (UniqueName: \"kubernetes.io/projected/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-kube-api-access-hbt84\") pod \"rate-test-model-c04a64-kserve-97fbc95-nppgk\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.489887 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.489795 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:04:54.626393 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.626351 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk"] Jul 03 00:04:54.628434 ip-10-0-137-14 kubenswrapper[2555]: W0703 00:04:54.628400 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f9d1fd4_9d7d_4cbe_a13e_2784392f1266.slice/crio-c4301e61cf476974b9543bdcc5c6057481afc7bcacf9923585bb873cf4146853 WatchSource:0}: Error finding container c4301e61cf476974b9543bdcc5c6057481afc7bcacf9923585bb873cf4146853: Status 404 returned error can't find the container with id c4301e61cf476974b9543bdcc5c6057481afc7bcacf9923585bb873cf4146853 Jul 03 00:04:54.882798 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.882760 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" event={"ID":"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266","Type":"ContainerStarted","Data":"f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b"} Jul 03 00:04:54.883011 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.882808 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" event={"ID":"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266","Type":"ContainerStarted","Data":"c4301e61cf476974b9543bdcc5c6057481afc7bcacf9923585bb873cf4146853"} Jul 03 00:04:54.898061 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:54.898007 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" podStartSLOduration=0.897988925 podStartE2EDuration="897.988925ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:04:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:04:54.896809058 +0000 UTC m=+2541.636101517" watchObservedRunningTime="2026-07-03 00:04:54.897988925 +0000 UTC m=+2541.637281382" Jul 03 00:04:58.140972 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:04:58.140932 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:05:04.490413 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:04.490355 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:05:04.490413 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:04.490423 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:05:04.502941 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:04.502916 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:05:04.942001 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:04.941969 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:05:19.835762 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:19.835728 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:05:32.439632 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:32.439591 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:05:39.407202 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:39.407168 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst"] Jul 03 00:05:40.262729 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:40.262687 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk"] Jul 03 00:05:44.752387 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:44.752330 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:05:54.428927 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.428817 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" podUID="bd121117-bd04-4d57-9fd7-1c0df52d4c1c" containerName="main" containerID="cri-o://06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e" gracePeriod=45 Jul 03 00:05:54.678242 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.678213 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:05:54.730938 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.730853 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-model-cache\") pod \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " Jul 03 00:05:54.730938 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.730889 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-home\") pod \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " Jul 03 00:05:54.730938 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.730936 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-dshm\") pod \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " Jul 03 00:05:54.731245 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.730962 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tmp-dir\") pod \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " Jul 03 00:05:54.731245 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.730978 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpkwp\" (UniqueName: \"kubernetes.io/projected/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-kube-api-access-vpkwp\") pod \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " Jul 03 00:05:54.731245 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.731041 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tls-certs\") pod \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\" (UID: \"bd121117-bd04-4d57-9fd7-1c0df52d4c1c\") " Jul 03 00:05:54.731245 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.731171 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd121117-bd04-4d57-9fd7-1c0df52d4c1c" (UID: "bd121117-bd04-4d57-9fd7-1c0df52d4c1c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:05:54.731245 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.731221 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd121117-bd04-4d57-9fd7-1c0df52d4c1c" (UID: "bd121117-bd04-4d57-9fd7-1c0df52d4c1c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:05:54.731512 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.731259 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-home" (OuterVolumeSpecName: "home") pod "bd121117-bd04-4d57-9fd7-1c0df52d4c1c" (UID: "bd121117-bd04-4d57-9fd7-1c0df52d4c1c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:05:54.731512 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.731432 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-model-cache\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:54.731512 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.731453 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-home\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:54.731512 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.731466 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tmp-dir\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:54.733314 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.733284 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-dshm" (OuterVolumeSpecName: "dshm") pod "bd121117-bd04-4d57-9fd7-1c0df52d4c1c" (UID: "bd121117-bd04-4d57-9fd7-1c0df52d4c1c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:05:54.733314 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.733289 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd121117-bd04-4d57-9fd7-1c0df52d4c1c" (UID: "bd121117-bd04-4d57-9fd7-1c0df52d4c1c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 00:05:54.733492 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.733389 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-kube-api-access-vpkwp" (OuterVolumeSpecName: "kube-api-access-vpkwp") pod "bd121117-bd04-4d57-9fd7-1c0df52d4c1c" (UID: "bd121117-bd04-4d57-9fd7-1c0df52d4c1c"). InnerVolumeSpecName "kube-api-access-vpkwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:05:54.832609 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.832570 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-dshm\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:54.832609 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.832600 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpkwp\" (UniqueName: \"kubernetes.io/projected/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-kube-api-access-vpkwp\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:54.832609 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:54.832611 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd121117-bd04-4d57-9fd7-1c0df52d4c1c-tls-certs\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:55.177559 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.177522 2555 generic.go:356] "Generic (PLEG): container finished" podID="bd121117-bd04-4d57-9fd7-1c0df52d4c1c" containerID="06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e" exitCode=0 Jul 03 00:05:55.177559 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.177557 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" Jul 03 00:05:55.177787 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.177570 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" event={"ID":"bd121117-bd04-4d57-9fd7-1c0df52d4c1c","Type":"ContainerDied","Data":"06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e"} Jul 03 00:05:55.177787 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.177607 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst" event={"ID":"bd121117-bd04-4d57-9fd7-1c0df52d4c1c","Type":"ContainerDied","Data":"2c20658ea94e0eb6239ac39fc65f11d3e1384ac01d3281bc56237f5b8a531377"} Jul 03 00:05:55.177787 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.177625 2555 scope.go:117] "RemoveContainer" containerID="06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e" Jul 03 00:05:55.189611 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.189589 2555 scope.go:117] "RemoveContainer" containerID="06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e" Jul 03 00:05:55.189921 ip-10-0-137-14 kubenswrapper[2555]: E0703 00:05:55.189900 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e\": container with ID starting with 06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e not found: ID does not exist" containerID="06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e" Jul 03 00:05:55.189996 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.189929 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e"} err="failed to get container status \"06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e\": rpc error: code = NotFound desc = could not find container \"06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e\": container with ID starting with 06b63f47443b0ebfa11d610c1c25f06d756c542a8c06193417e7d2de0a541a7e not found: ID does not exist" Jul 03 00:05:55.199641 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.199615 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst"] Jul 03 00:05:55.205301 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.205279 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/rate-test-model-c1893b-kserve-7f56c44f87-9ljst"] Jul 03 00:05:55.284198 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.284136 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" podUID="1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" containerName="main" containerID="cri-o://f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b" gracePeriod=45 Jul 03 00:05:55.535994 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.535966 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:05:55.639859 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.639816 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-model-cache\") pod \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " Jul 03 00:05:55.639859 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.639858 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-dshm\") pod \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " Jul 03 00:05:55.640100 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.639889 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tmp-dir\") pod \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " Jul 03 00:05:55.640100 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.639914 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbt84\" (UniqueName: \"kubernetes.io/projected/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-kube-api-access-hbt84\") pod \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " Jul 03 00:05:55.640100 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.639931 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tls-certs\") pod \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " Jul 03 00:05:55.640100 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.639949 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-home\") pod \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\" (UID: \"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266\") " Jul 03 00:05:55.640314 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.640139 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" (UID: "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:05:55.640314 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.640196 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" (UID: "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:05:55.640452 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.640336 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-home" (OuterVolumeSpecName: "home") pod "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" (UID: "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:05:55.640452 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.640387 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-model-cache\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:55.640452 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.640405 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tmp-dir\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:55.642221 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.642199 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-dshm" (OuterVolumeSpecName: "dshm") pod "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" (UID: "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:05:55.642327 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.642246 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-kube-api-access-hbt84" (OuterVolumeSpecName: "kube-api-access-hbt84") pod "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" (UID: "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266"). InnerVolumeSpecName "kube-api-access-hbt84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:05:55.642327 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.642267 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" (UID: "1f9d1fd4-9d7d-4cbe-a13e-2784392f1266"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 00:05:55.683146 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.683109 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd121117-bd04-4d57-9fd7-1c0df52d4c1c" path="/var/lib/kubelet/pods/bd121117-bd04-4d57-9fd7-1c0df52d4c1c/volumes" Jul 03 00:05:55.741478 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.741390 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-dshm\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:55.741478 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.741418 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbt84\" (UniqueName: \"kubernetes.io/projected/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-kube-api-access-hbt84\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:55.741478 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.741429 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-tls-certs\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:55.741478 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:55.741438 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266-home\") on node \"ip-10-0-137-14.ec2.internal\" DevicePath \"\"" Jul 03 00:05:56.183439 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.183402 2555 generic.go:356] "Generic (PLEG): container finished" podID="1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" containerID="f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b" exitCode=0 Jul 03 00:05:56.183621 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.183440 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" event={"ID":"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266","Type":"ContainerDied","Data":"f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b"} Jul 03 00:05:56.183621 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.183492 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" event={"ID":"1f9d1fd4-9d7d-4cbe-a13e-2784392f1266","Type":"ContainerDied","Data":"c4301e61cf476974b9543bdcc5c6057481afc7bcacf9923585bb873cf4146853"} Jul 03 00:05:56.183621 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.183516 2555 scope.go:117] "RemoveContainer" containerID="f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b" Jul 03 00:05:56.183621 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.183453 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk" Jul 03 00:05:56.192132 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.192116 2555 scope.go:117] "RemoveContainer" containerID="f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b" Jul 03 00:05:56.192401 ip-10-0-137-14 kubenswrapper[2555]: E0703 00:05:56.192385 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b\": container with ID starting with f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b not found: ID does not exist" containerID="f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b" Jul 03 00:05:56.192460 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.192409 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b"} err="failed to get container status \"f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b\": rpc error: code = NotFound desc = could not find container \"f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b\": container with ID starting with f090c67ef76510a224ce117d384e8f171090f72484e906164773a5f6f0d4134b not found: ID does not exist" Jul 03 00:05:56.198384 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.198335 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk"] Jul 03 00:05:56.201242 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:56.201218 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c04a64/rate-test-model-c04a64-kserve-97fbc95-nppgk"] Jul 03 00:05:57.683397 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:05:57.683343 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" path="/var/lib/kubelet/pods/1f9d1fd4-9d7d-4cbe-a13e-2784392f1266/volumes" Jul 03 00:06:28.136521 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:28.136482 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xdz5w"] Jul 03 00:06:37.867489 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:37.867455 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f45646c76-l5nkm_230f226f-a085-4b9c-94d1-6810b0c40c1c/copy-manifests/0.log" Jul 03 00:06:37.874601 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:37.874576 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f45646c76-l5nkm_230f226f-a085-4b9c-94d1-6810b0c40c1c/manager/0.log" Jul 03 00:06:39.677231 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:39.677192 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zc4l6_e53ff879-1c33-4789-bedc-603bc7fc8d7b/kuadrant-console-plugin/0.log" Jul 03 00:06:40.046540 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:40.046514 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xdz5w_4854edda-01da-493c-b0bd-d6735e5524bc/limitador/0.log" Jul 03 00:06:40.771534 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:40.771501 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-576b444b4-8p2gf_a3f5372d-6d8e-4b44-bf5f-56f43829b580/kube-auth-proxy/0.log" Jul 03 00:06:42.263724 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:42.263695 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-dwffk_42e3d0db-1714-43f5-b280-54378eb86769/main/0.log" Jul 03 00:06:42.408283 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:42.408249 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-2k457_4e6c4ed3-6fa7-4ae1-b5ee-85635da25a88/main/0.log" Jul 03 00:06:47.359447 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.359411 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x5drt/must-gather-tv6pn"] Jul 03 00:06:47.359788 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.359750 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd121117-bd04-4d57-9fd7-1c0df52d4c1c" containerName="main" Jul 03 00:06:47.359788 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.359762 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd121117-bd04-4d57-9fd7-1c0df52d4c1c" containerName="main" Jul 03 00:06:47.359788 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.359782 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" containerName="main" Jul 03 00:06:47.359788 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.359787 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" containerName="main" Jul 03 00:06:47.359917 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.359838 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9d1fd4-9d7d-4cbe-a13e-2784392f1266" containerName="main" Jul 03 00:06:47.359917 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.359849 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd121117-bd04-4d57-9fd7-1c0df52d4c1c" containerName="main" Jul 03 00:06:47.362727 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.362710 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x5drt/must-gather-tv6pn" Jul 03 00:06:47.365166 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.365149 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x5drt\"/\"kube-root-ca.crt\"" Jul 03 00:06:47.366124 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.366090 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x5drt\"/\"openshift-service-ca.crt\"" Jul 03 00:06:47.366124 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.366089 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x5drt\"/\"default-dockercfg-cjp6x\"" Jul 03 00:06:47.373075 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.373054 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x5drt/must-gather-tv6pn"] Jul 03 00:06:47.487792 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.487756 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbxnp\" (UniqueName: \"kubernetes.io/projected/fe263ca1-9d5f-427b-9bcc-38bf47c36bc0-kube-api-access-gbxnp\") pod \"must-gather-tv6pn\" (UID: \"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0\") " pod="openshift-must-gather-x5drt/must-gather-tv6pn" Jul 03 00:06:47.487936 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.487800 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fe263ca1-9d5f-427b-9bcc-38bf47c36bc0-must-gather-output\") pod \"must-gather-tv6pn\" (UID: \"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0\") " pod="openshift-must-gather-x5drt/must-gather-tv6pn" Jul 03 00:06:47.592385 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.589772 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbxnp\" (UniqueName: \"kubernetes.io/projected/fe263ca1-9d5f-427b-9bcc-38bf47c36bc0-kube-api-access-gbxnp\") pod \"must-gather-tv6pn\" (UID: \"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0\") " pod="openshift-must-gather-x5drt/must-gather-tv6pn" Jul 03 00:06:47.592385 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.589834 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fe263ca1-9d5f-427b-9bcc-38bf47c36bc0-must-gather-output\") pod \"must-gather-tv6pn\" (UID: \"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0\") " pod="openshift-must-gather-x5drt/must-gather-tv6pn" Jul 03 00:06:47.592385 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.590221 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fe263ca1-9d5f-427b-9bcc-38bf47c36bc0-must-gather-output\") pod \"must-gather-tv6pn\" (UID: \"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0\") " pod="openshift-must-gather-x5drt/must-gather-tv6pn" Jul 03 00:06:47.602056 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.602030 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbxnp\" (UniqueName: \"kubernetes.io/projected/fe263ca1-9d5f-427b-9bcc-38bf47c36bc0-kube-api-access-gbxnp\") pod \"must-gather-tv6pn\" (UID: \"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0\") " pod="openshift-must-gather-x5drt/must-gather-tv6pn" Jul 03 00:06:47.672862 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.672802 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x5drt/must-gather-tv6pn" Jul 03 00:06:47.800301 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.800276 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x5drt/must-gather-tv6pn"] Jul 03 00:06:47.802306 ip-10-0-137-14 kubenswrapper[2555]: W0703 00:06:47.802277 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe263ca1_9d5f_427b_9bcc_38bf47c36bc0.slice/crio-1539c213cf1f71ec3f5a1af47ccc8fef3a5244488589b9aa05c408b84a06aaa3 WatchSource:0}: Error finding container 1539c213cf1f71ec3f5a1af47ccc8fef3a5244488589b9aa05c408b84a06aaa3: Status 404 returned error can't find the container with id 1539c213cf1f71ec3f5a1af47ccc8fef3a5244488589b9aa05c408b84a06aaa3 Jul 03 00:06:47.804062 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:47.804046 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 00:06:48.426194 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:48.426158 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x5drt/must-gather-tv6pn" event={"ID":"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0","Type":"ContainerStarted","Data":"1539c213cf1f71ec3f5a1af47ccc8fef3a5244488589b9aa05c408b84a06aaa3"} Jul 03 00:06:49.432819 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:49.432784 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x5drt/must-gather-tv6pn" event={"ID":"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0","Type":"ContainerStarted","Data":"d9ba2b299343e7093488a8c312b3dd55ad985f8ceab6bfcda979c086f0fccb14"} Jul 03 00:06:49.432819 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:49.432826 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x5drt/must-gather-tv6pn" event={"ID":"fe263ca1-9d5f-427b-9bcc-38bf47c36bc0","Type":"ContainerStarted","Data":"3555ddb884f01de96d92733b2808baa93cb34ac49269b18d809c36d259d4e722"} Jul 03 00:06:49.448320 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:49.448236 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x5drt/must-gather-tv6pn" podStartSLOduration=1.589867232 podStartE2EDuration="2.448215328s" totalImagesPullingTime="858.348096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:06:49.447264926 +0000 UTC m=+2656.186557385" watchObservedRunningTime="2026-07-03 00:06:49.448215328 +0000 UTC m=+2656.187507767" Jul 03 00:06:50.119541 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:50.119508 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hhcrv_c1b6e71e-c320-46f2-b9f7-07fd0a76cabc/global-pull-secret-syncer/0.log" Jul 03 00:06:50.320074 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:50.320043 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kknfx_67cf61d5-5693-41f5-8555-de6b76bd89a5/konnectivity-agent/0.log" Jul 03 00:06:50.390749 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:50.390659 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-14.ec2.internal_394edcdb51bee6f5516971692e04a798/haproxy/0.log" Jul 03 00:06:55.961913 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:55.961874 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zc4l6_e53ff879-1c33-4789-bedc-603bc7fc8d7b/kuadrant-console-plugin/0.log" Jul 03 00:06:56.114672 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:56.114641 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xdz5w_4854edda-01da-493c-b0bd-d6735e5524bc/limitador/0.log" Jul 03 00:06:57.877211 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:57.877135 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b579166-b977-4c2d-9eb9-250e2748c634/alertmanager/0.log" Jul 03 00:06:57.899598 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:57.899571 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b579166-b977-4c2d-9eb9-250e2748c634/config-reloader/0.log" Jul 03 00:06:57.921092 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:57.921057 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b579166-b977-4c2d-9eb9-250e2748c634/kube-rbac-proxy-web/0.log" Jul 03 00:06:57.944558 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:57.944522 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b579166-b977-4c2d-9eb9-250e2748c634/kube-rbac-proxy/0.log" Jul 03 00:06:57.967322 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:57.967291 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b579166-b977-4c2d-9eb9-250e2748c634/kube-rbac-proxy-metric/0.log" Jul 03 00:06:57.997347 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:57.997319 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b579166-b977-4c2d-9eb9-250e2748c634/prom-label-proxy/0.log" Jul 03 00:06:58.023934 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.023909 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b579166-b977-4c2d-9eb9-250e2748c634/init-config-reloader/0.log" Jul 03 00:06:58.085670 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.085633 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-s6zw4_bfb829dd-f5e9-4066-84c1-edf3440398cc/kube-state-metrics/0.log" Jul 03 00:06:58.102009 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.101982 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-s6zw4_bfb829dd-f5e9-4066-84c1-edf3440398cc/kube-rbac-proxy-main/0.log" Jul 03 00:06:58.123274 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.123244 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-s6zw4_bfb829dd-f5e9-4066-84c1-edf3440398cc/kube-rbac-proxy-self/0.log" Jul 03 00:06:58.154450 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.154422 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-58ffd5b7cd-9vl2m_ba025c2d-86d1-4d5d-b6db-246e60d7ec24/metrics-server/0.log" Jul 03 00:06:58.217527 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.217496 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2f8gc_b018168b-5e45-4312-bf72-06c8e9fdbb9d/node-exporter/0.log" Jul 03 00:06:58.238759 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.238721 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2f8gc_b018168b-5e45-4312-bf72-06c8e9fdbb9d/kube-rbac-proxy/0.log" Jul 03 00:06:58.257961 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.257934 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2f8gc_b018168b-5e45-4312-bf72-06c8e9fdbb9d/init-textfile/0.log" Jul 03 00:06:58.688828 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.688796 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-ptpbp_2f28e20a-a3e6-4989-9b5f-a87f1b3669c9/prometheus-operator-admission-webhook/0.log" Jul 03 00:06:58.717312 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.717276 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55b9649fcd-v6x5t_31093d2e-124f-44be-a7d4-10dc1259a965/telemeter-client/0.log" Jul 03 00:06:58.743142 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.743113 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55b9649fcd-v6x5t_31093d2e-124f-44be-a7d4-10dc1259a965/reload/0.log" Jul 03 00:06:58.763678 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.763654 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55b9649fcd-v6x5t_31093d2e-124f-44be-a7d4-10dc1259a965/kube-rbac-proxy/0.log" Jul 03 00:06:58.792503 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.792473 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66b64698c7-m6k4d_5061fb7d-1dfa-4671-aadd-523c08eb4822/thanos-query/0.log" Jul 03 00:06:58.812157 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.812129 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66b64698c7-m6k4d_5061fb7d-1dfa-4671-aadd-523c08eb4822/kube-rbac-proxy-web/0.log" Jul 03 00:06:58.831513 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.831473 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66b64698c7-m6k4d_5061fb7d-1dfa-4671-aadd-523c08eb4822/kube-rbac-proxy/0.log" Jul 03 00:06:58.850410 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.850344 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66b64698c7-m6k4d_5061fb7d-1dfa-4671-aadd-523c08eb4822/prom-label-proxy/0.log" Jul 03 00:06:58.870451 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.870419 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66b64698c7-m6k4d_5061fb7d-1dfa-4671-aadd-523c08eb4822/kube-rbac-proxy-rules/0.log" Jul 03 00:06:58.890783 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:58.890749 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66b64698c7-m6k4d_5061fb7d-1dfa-4671-aadd-523c08eb4822/kube-rbac-proxy-metrics/0.log" Jul 03 00:06:59.887715 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:59.887672 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc"] Jul 03 00:06:59.896011 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:59.895985 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:06:59.899962 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:06:59.899930 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc"] Jul 03 00:07:00.029159 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.029107 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-lib-modules\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.029343 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.029174 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-podres\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.029343 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.029274 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9f6k\" (UniqueName: \"kubernetes.io/projected/0cc6f6a0-aa64-4257-94d0-250e41cbad74-kube-api-access-s9f6k\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.029343 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.029309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-sys\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.029521 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.029398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-proc\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.130913 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.130876 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-lib-modules\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.130913 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.130914 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-podres\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.131147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.130959 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9f6k\" (UniqueName: \"kubernetes.io/projected/0cc6f6a0-aa64-4257-94d0-250e41cbad74-kube-api-access-s9f6k\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.131147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.130977 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-sys\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.131147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.131011 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-proc\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.131147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.131083 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-lib-modules\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.131147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.131101 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-proc\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.131147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.131101 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-sys\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.131147 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.131083 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0cc6f6a0-aa64-4257-94d0-250e41cbad74-podres\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.138434 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.138351 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9f6k\" (UniqueName: \"kubernetes.io/projected/0cc6f6a0-aa64-4257-94d0-250e41cbad74-kube-api-access-s9f6k\") pod \"perf-node-gather-daemonset-4f8cc\" (UID: \"0cc6f6a0-aa64-4257-94d0-250e41cbad74\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.212676 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.212633 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:00.400301 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.398012 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc"] Jul 03 00:07:00.400623 ip-10-0-137-14 kubenswrapper[2555]: W0703 00:07:00.400580 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0cc6f6a0_aa64_4257_94d0_250e41cbad74.slice/crio-4caee08a2de05f2944a0e5363069eb0d3c9256dff22f6bf97ee93cb28e3047c5 WatchSource:0}: Error finding container 4caee08a2de05f2944a0e5363069eb0d3c9256dff22f6bf97ee93cb28e3047c5: Status 404 returned error can't find the container with id 4caee08a2de05f2944a0e5363069eb0d3c9256dff22f6bf97ee93cb28e3047c5 Jul 03 00:07:00.512179 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:00.512096 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" event={"ID":"0cc6f6a0-aa64-4257-94d0-250e41cbad74","Type":"ContainerStarted","Data":"4caee08a2de05f2944a0e5363069eb0d3c9256dff22f6bf97ee93cb28e3047c5"} Jul 03 00:07:01.023173 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:01.023143 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-x469x_feec3cd6-90b3-4633-b539-3a4a8c827706/download-server/0.log" Jul 03 00:07:01.519910 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:01.519876 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" event={"ID":"0cc6f6a0-aa64-4257-94d0-250e41cbad74","Type":"ContainerStarted","Data":"8682d7bbea075225ccc51b428b0920a0212dd49b02c8235a16f423944d782a26"} Jul 03 00:07:01.520114 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:01.520005 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:01.534553 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:01.534507 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" podStartSLOduration=2.5344921769999997 podStartE2EDuration="2.534492177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 00:06:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:07:01.533258065 +0000 UTC m=+2668.272550543" watchObservedRunningTime="2026-07-03 00:07:01.534492177 +0000 UTC m=+2668.273784654" Jul 03 00:07:02.529937 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:02.529895 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cgggg_a3c931cb-95f1-4dc4-8730-204905465bcf/dns/0.log" Jul 03 00:07:02.548908 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:02.548877 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cgggg_a3c931cb-95f1-4dc4-8730-204905465bcf/kube-rbac-proxy/0.log" Jul 03 00:07:02.653077 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:02.653052 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qpvc2_2dfdc72a-9586-4e0b-b68d-7008640b7770/dns-node-resolver/0.log" Jul 03 00:07:03.174203 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:03.174164 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-59c99698c5-rnqdk_d03b71c8-8d20-482f-a075-21ab7ae4976d/registry/0.log" Jul 03 00:07:03.198323 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:03.198296 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pnb5p_401fe81a-d58d-430b-bb94-f73fdce48999/node-ca/0.log" Jul 03 00:07:04.237270 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:04.237242 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-576b444b4-8p2gf_a3f5372d-6d8e-4b44-bf5f-56f43829b580/kube-auth-proxy/0.log" Jul 03 00:07:05.153797 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:05.153753 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbp86_070538f1-749e-45d0-bf8d-f3219633dd6d/serve-healthcheck-canary/0.log" Jul 03 00:07:05.688269 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:05.688237 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b7245_a296e165-7e89-4db7-8e0b-c86cea87bf46/kube-rbac-proxy/0.log" Jul 03 00:07:05.709888 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:05.709850 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b7245_a296e165-7e89-4db7-8e0b-c86cea87bf46/exporter/0.log" Jul 03 00:07:05.731865 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:05.731836 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b7245_a296e165-7e89-4db7-8e0b-c86cea87bf46/extractor/0.log" Jul 03 00:07:07.540120 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:07.540086 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-4f8cc" Jul 03 00:07:08.410432 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:08.410254 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f45646c76-l5nkm_230f226f-a085-4b9c-94d1-6810b0c40c1c/manager/0.log" Jul 03 00:07:08.433648 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:08.433618 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f45646c76-l5nkm_230f226f-a085-4b9c-94d1-6810b0c40c1c/copy-manifests/0.log" Jul 03 00:07:10.004930 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:10.004889 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-679bd8f54d-mt42q_821ce603-37b3-4e65-b559-83f2ec1b7ef2/manager/0.log" Jul 03 00:07:10.011463 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:10.011439 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-576b444b4-8p2gf_a3f5372d-6d8e-4b44-bf5f-56f43829b580/kube-auth-proxy/0.log" Jul 03 00:07:10.029762 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:10.029733 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hvwbl_eb7681ba-c381-4453-9f66-ccdf717bfff2/openshift-lws-operator/0.log" Jul 03 00:07:11.482795 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:11.482767 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-576b444b4-8p2gf_a3f5372d-6d8e-4b44-bf5f-56f43829b580/kube-auth-proxy/0.log" Jul 03 00:07:14.621611 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:14.621580 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-57bsf_6aaac3b1-ed0e-44b7-be54-0aecd1f3b527/migrator/0.log" Jul 03 00:07:14.640156 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:14.640124 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-57bsf_6aaac3b1-ed0e-44b7-be54-0aecd1f3b527/graceful-termination/0.log" Jul 03 00:07:16.211052 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.211027 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrhzq_507bb56b-96aa-4ab3-b284-b589f3959f4e/kube-multus-additional-cni-plugins/0.log" Jul 03 00:07:16.230319 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.230292 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrhzq_507bb56b-96aa-4ab3-b284-b589f3959f4e/egress-router-binary-copy/0.log" Jul 03 00:07:16.248585 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.248563 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrhzq_507bb56b-96aa-4ab3-b284-b589f3959f4e/cni-plugins/0.log" Jul 03 00:07:16.266612 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.266594 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrhzq_507bb56b-96aa-4ab3-b284-b589f3959f4e/bond-cni-plugin/0.log" Jul 03 00:07:16.285032 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.285016 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrhzq_507bb56b-96aa-4ab3-b284-b589f3959f4e/routeoverride-cni/0.log" Jul 03 00:07:16.303493 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.303472 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrhzq_507bb56b-96aa-4ab3-b284-b589f3959f4e/whereabouts-cni-bincopy/0.log" Jul 03 00:07:16.321493 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.321472 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qrhzq_507bb56b-96aa-4ab3-b284-b589f3959f4e/whereabouts-cni/0.log" Jul 03 00:07:16.388330 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.388305 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nlbp4_7bdff9fb-c404-400f-9db5-c432287ed695/kube-multus/0.log" Jul 03 00:07:16.447410 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.447381 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-88h28_f4cf293e-362f-45b5-aad9-e3a34a8f52f8/network-metrics-daemon/0.log" Jul 03 00:07:16.465389 ip-10-0-137-14 kubenswrapper[2555]: I0703 00:07:16.465253 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-88h28_f4cf293e-362f-45b5-aad9-e3a34a8f52f8/kube-rbac-proxy/0.log"