Jul 08 17:37:03.712522 ip-10-0-129-217 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 17:37:03.712534 ip-10-0-129-217 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 17:37:03.712540 ip-10-0-129-217 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 17:37:03.712777 ip-10-0-129-217 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 17:37:13.960933 ip-10-0-129-217 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 17:37:13.960949 ip-10-0-129-217 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b07ba3868c974e58b61b4762686fbe4e -- Jul 08 17:39:10.827840 ip-10-0-129-217 systemd[1]: Starting Kubernetes Kubelet... Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.213038 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218021 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218037 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218041 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218044 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218047 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218050 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218053 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:39:11.274442 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218057 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218060 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218063 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218066 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218068 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218071 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218074 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218077 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218080 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218084 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218087 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218090 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218094 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218096 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218099 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218103 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218106 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218110 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218114 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218118 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:39:11.275513 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218121 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218124 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218127 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218130 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218132 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218135 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218138 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218141 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218144 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218146 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218149 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218152 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218154 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218157 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218159 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218162 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218165 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218168 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218171 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:39:11.276554 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218174 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218177 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218179 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218182 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218184 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218187 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218190 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218194 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218198 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218201 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218204 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218207 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218210 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218213 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218216 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218219 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218221 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218224 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218228 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:39:11.277189 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218232 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218235 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218238 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218241 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218243 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218246 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218248 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218251 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218254 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218257 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218260 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218263 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218266 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218269 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218275 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218278 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218281 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218284 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218287 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218289 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:39:11.278229 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218293 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218295 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218298 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218301 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218304 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218307 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218310 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218312 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218315 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218318 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218321 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218323 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218326 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.218329 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219064 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219071 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219074 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219077 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219080 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219083 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:39:11.279116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219086 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219088 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219091 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219094 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219096 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219099 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219102 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219106 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219108 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219113 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219117 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219121 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219123 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219126 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219129 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219132 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219134 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219137 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219139 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:39:11.280843 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219143 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219145 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219148 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219151 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219153 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219158 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219161 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219164 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219168 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219171 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219174 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219177 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219179 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219182 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219185 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219188 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219190 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219193 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219195 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:39:11.281477 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219198 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219201 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219204 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219206 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219209 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219213 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219215 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219218 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219221 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219223 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219225 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219228 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219231 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219233 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219236 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219238 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219241 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219243 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219246 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219249 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219251 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:39:11.282111 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219254 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219257 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219259 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219262 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219264 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219267 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219270 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219272 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219275 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219278 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219280 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219283 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219286 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219290 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219295 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219297 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219301 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219305 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219308 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:39:11.282740 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219311 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219314 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219317 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219320 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219322 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219325 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219328 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219331 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219334 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219337 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219340 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219343 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219345 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219348 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219351 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219738 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219742 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219745 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219748 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219753 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:39:11.283798 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219756 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219759 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219761 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219764 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219767 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219770 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219773 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219776 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219781 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219785 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219788 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219791 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219794 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219796 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219799 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219802 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219805 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219808 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219811 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:39:11.284429 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219814 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219817 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219819 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219822 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219825 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219828 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219831 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219833 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219836 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219838 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219841 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219844 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219848 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219850 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219854 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219856 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219859 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219862 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219864 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219867 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:39:11.285493 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219869 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219872 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219875 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219878 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219880 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219883 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219886 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219889 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219893 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219895 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219898 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219900 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219903 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219906 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219909 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219911 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219913 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219916 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219920 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:39:11.286122 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219924 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219927 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219930 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219933 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219936 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219940 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219943 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219945 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219949 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219953 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219955 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219958 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219961 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219964 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219966 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219970 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219973 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219975 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219979 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:39:11.286704 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219981 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219984 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219987 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219990 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219992 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219995 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.219997 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220000 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220002 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220005 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220007 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220010 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220012 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220015 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220018 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220020 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.220023 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220101 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220109 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220117 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220127 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 17:39:11.287246 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220131 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220135 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220139 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220143 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220147 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220150 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220153 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220156 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220161 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220165 2580 flags.go:64] FLAG: --cgroup-root="" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220168 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220171 2580 flags.go:64] FLAG: --client-ca-file="" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220174 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220177 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220181 2580 flags.go:64] FLAG: --cluster-domain="" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220184 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220187 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220190 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220193 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220197 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220200 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220203 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220207 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220210 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 08 17:39:11.287870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220213 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220216 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220219 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220222 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220226 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220231 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220234 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220237 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220240 2580 flags.go:64] FLAG: --enable-server="true" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220243 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220889 2580 flags.go:64] FLAG: --event-burst="100" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220893 2580 flags.go:64] FLAG: --event-qps="50" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220899 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220902 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220905 2580 flags.go:64] FLAG: --eviction-hard="" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220910 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220913 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220917 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220920 2580 flags.go:64] FLAG: --eviction-soft="" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220923 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220926 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220929 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220932 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220935 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220937 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 17:39:11.288520 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220940 2580 flags.go:64] FLAG: --feature-gates="" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220944 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220947 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220950 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220954 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220957 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220960 2580 flags.go:64] FLAG: --help="false" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220963 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220966 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220969 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220972 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220976 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220980 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220983 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220986 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220989 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220993 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220996 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.220999 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221003 2580 flags.go:64] FLAG: --kube-reserved="" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221006 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221009 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221012 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221015 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 17:39:11.289431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221017 2580 flags.go:64] FLAG: --lock-file="" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221021 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221024 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221027 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221033 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221036 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221038 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221041 2580 flags.go:64] FLAG: --logging-format="text" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221044 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221047 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221050 2580 flags.go:64] FLAG: --manifest-url="" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221053 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221057 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221061 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221065 2580 flags.go:64] FLAG: --max-pods="110" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221068 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221071 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221074 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221077 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221080 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221085 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221088 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221095 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221098 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221101 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 17:39:11.290153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221104 2580 flags.go:64] FLAG: --pod-cidr="" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221107 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221114 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221117 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221120 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221123 2580 flags.go:64] FLAG: --port="10250" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221126 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221129 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0299cc344ae4b1e43" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221133 2580 flags.go:64] FLAG: --qos-reserved="" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221136 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221139 2580 flags.go:64] FLAG: --register-node="true" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221142 2580 flags.go:64] FLAG: --register-with-taints="" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221146 2580 flags.go:64] FLAG: --registry-burst="10" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221149 2580 flags.go:64] FLAG: --registry-qps="5" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221151 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221154 2580 flags.go:64] FLAG: --reserved-memory="" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221158 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221162 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221165 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221167 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221170 2580 flags.go:64] FLAG: --runonce="false" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221173 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221176 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221179 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221182 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 17:39:11.290952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221185 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221188 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221192 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221196 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221199 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221202 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221205 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221208 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221211 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221215 2580 flags.go:64] FLAG: --system-cgroups="" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221218 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221223 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221226 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221229 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221233 2580 flags.go:64] FLAG: --tls-min-version="" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221236 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221239 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221243 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221246 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221250 2580 flags.go:64] FLAG: --v="2" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221254 2580 flags.go:64] FLAG: --version="false" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221258 2580 flags.go:64] FLAG: --vmodule="" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221262 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.221265 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221399 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:39:11.291755 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221403 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221407 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221410 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221413 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221416 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221418 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221421 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221424 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221426 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221429 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221433 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221435 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221438 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221441 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221444 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221446 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221450 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221453 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221456 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221459 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:39:11.292457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221462 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221465 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221468 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221470 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221473 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221476 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221479 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221482 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221484 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221487 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221489 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221492 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221494 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221497 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221500 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221502 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221505 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221507 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221510 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221512 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:39:11.293133 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221515 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221518 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221522 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221525 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221527 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221530 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221532 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221535 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221540 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221542 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221545 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221548 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221552 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221555 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221558 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221561 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221564 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221579 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221582 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:39:11.293691 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221586 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221589 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221591 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221596 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221599 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221603 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221606 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221610 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221613 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221616 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221618 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221621 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221624 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221626 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221635 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221640 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221642 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221645 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221648 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:39:11.294219 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221650 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221653 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221658 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221660 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221663 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221666 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221668 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221671 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221674 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221676 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221679 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221682 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221684 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221688 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221691 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221693 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221696 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221699 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221701 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:39:11.295140 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.221705 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.222348 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.229562 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.229596 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.229619 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.229628 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229697 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229703 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229706 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229709 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229712 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229715 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229718 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229721 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229724 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:39:11.295788 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229727 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229730 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229733 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229735 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229739 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229741 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229744 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229747 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229749 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229752 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229754 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229757 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229760 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229762 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229765 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229767 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229770 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229774 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229776 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229782 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:39:11.362344 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229785 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:39:11.301917 ip-10-0-129-217 systemd[1]: Started Kubernetes Kubelet. Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229788 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229791 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229794 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229796 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229799 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229802 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229804 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229808 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229810 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229813 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229815 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229818 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229821 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229823 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229826 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229828 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229831 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229834 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:39:11.363311 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229837 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229839 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229842 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229845 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229847 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229850 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229853 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229855 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229858 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229861 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229863 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229866 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229870 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229873 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229876 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229879 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229881 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229884 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229887 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229889 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:39:11.364535 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229892 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229894 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229897 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229899 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229902 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229904 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229907 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229909 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229912 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229915 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229917 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229921 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229924 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229927 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229929 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229932 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229936 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229940 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229944 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:39:11.366234 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229947 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229950 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229953 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229955 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229958 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229963 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229967 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229970 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229973 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229976 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229978 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.229981 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.229986 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230107 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:39:11.367314 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230113 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230118 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230121 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230124 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230127 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230130 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230133 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230136 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230139 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230142 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230145 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230149 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230152 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230155 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230158 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230160 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230163 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230165 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230169 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230171 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:39:11.368400 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230174 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230177 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230180 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230183 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230186 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230189 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230191 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230195 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230197 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230200 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230203 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230205 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230208 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230211 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230213 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230216 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230218 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230221 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230223 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230226 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:39:11.369699 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230228 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230230 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230233 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230237 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230240 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230243 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230247 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230251 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230254 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230257 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230260 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230263 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230266 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230269 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230272 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230276 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230279 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230282 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230284 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:39:11.370729 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230287 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230290 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230293 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230295 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230298 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230300 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230303 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230306 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230308 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230310 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230313 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230315 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230318 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230320 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230323 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230326 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230330 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230333 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230336 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:39:11.372276 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230338 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230341 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230343 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230346 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230348 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230351 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230353 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230356 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230359 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230362 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230365 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230368 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230370 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230373 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230376 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230379 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230381 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230384 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230386 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:39:11.373246 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:11.230389 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.230394 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.230500 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.232832 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.234304 2580 server.go:1019] "Starting client certificate rotation" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.234413 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.234458 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.236763 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.238877 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.265011 2580 log.go:25] "Validated CRI v1 runtime API" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.268209 2580 log.go:25] "Validated CRI v1 image API" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.269691 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 17:39:11.374019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.273491 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 87b37769-a537-4aab-b18b-39befe6341d6:/dev/nvme0n1p3 8b118350-9a9b-4b11-af40-aa2f8d1eb2f0:/dev/nvme0n1p4] Jul 08 17:39:11.374433 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.273506 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 17:39:11.375284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.279778 2580 manager.go:216] Machine: {Timestamp:2026-07-08 17:39:11.278465919 +0000 UTC m=+0.341988743 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099957 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cce5bf9c22172baf2f8eada614b81 SystemUUID:ec2cce5b-f9c2-2172-baf2-f8eada614b81 BootID:b07ba386-8c97-4e58-b61b-4762686fbe4e Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cb:e4:2c:a2:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cb:e4:2c:a2:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:08:7d:71:bc:4d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 17:39:11.375284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.279883 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 17:39:11.375284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.279970 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 17:39:11.375284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.283045 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 17:39:11.375284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.283080 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-217.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 17:39:11.375284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.283869 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 17:39:11.375284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.283882 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 17:39:11.375284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.283891 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.285088 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.286766 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.289746 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.289868 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.290508 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.290526 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.290538 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.290548 2580 kubelet.go:398] "Adding apiserver pod source" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.290557 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.290924 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.290940 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.292691 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.292901 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.295306 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.295333 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296642 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296657 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296663 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296668 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296677 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296683 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296691 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296697 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296703 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296708 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296718 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.296724 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.298040 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.298055 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.301242 2580 server.go:1268] "Started kubelet" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.301655 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.303735 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.310359 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.310508 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.346987 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.347026 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k42sc" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.347044 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k42sc" Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.347306 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-217.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 17:39:11.375924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.347588 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.347612 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.348820 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.350416 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-217.ec2.internal.18c0613954b9f5e6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-217.ec2.internal,UID:ip-10-0-129-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-217.ec2.internal,},FirstTimestamp:2026-07-08 17:39:11.301219814 +0000 UTC m=+0.364742634,LastTimestamp:2026-07-08 17:39:11.301219814 +0000 UTC m=+0.364742634,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-217.ec2.internal,}" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.352954 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.352956 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.353028 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.353430 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.353444 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.353430 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.353797 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.353807 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.356009 2580 factory.go:55] Registering systemd factory Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.356025 2580 factory.go:223] Registration of the systemd container factory successfully Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.356249 2580 factory.go:153] Registering CRI-O factory Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.356262 2580 factory.go:223] Registration of the crio container factory successfully Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.356303 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.356322 2580 factory.go:103] Registering Raw factory Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.356335 2580 manager.go:1196] Started watching for new ooms in manager Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.356797 2580 manager.go:349] Starting recovery of all containers Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.357523 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.363201 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.364755 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-217.ec2.internal\" not found" node="ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.370044 2580 manager.go:354] Recovery completed Jul 08 17:39:11.377197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.377158 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:39:11.379361 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.378974 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:39:11.379361 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.379000 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:39:11.379361 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.379017 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:39:11.379972 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.379762 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 17:39:11.379972 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.379773 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 17:39:11.379972 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.379790 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 08 17:39:11.379972 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.379839 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.380713 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.380738 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.380753 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.380901 2580 policy_none.go:49] "None policy: Start" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.380914 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.380924 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.381825 2580 policy_none.go:47] "Start" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.432389 2580 manager.go:337] "Starting Device Plugin manager" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.432435 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.432450 2580 server.go:87] "Starting device plugin registration server" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.432764 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.432777 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.432923 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.433004 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.433013 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.433998 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 17:39:11.489214 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.434047 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:11.585126 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.533480 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:39:11.585126 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.534800 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:39:11.585126 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.534826 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:39:11.585126 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.534835 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:39:11.585126 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.534870 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.585126 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.544177 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.585126 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.544199 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-217.ec2.internal\": node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:11.585126 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.572821 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:11.674117 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.673548 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:11.804658 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.773921 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:11.840045 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.839967 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 17:39:11.840045 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.840032 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 17:39:11.840214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.840087 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 17:39:11.840214 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.840132 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 17:39:11.842858 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.842739 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:39:11.875083 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.875057 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:11.945645 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.945601 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal"] Jul 08 17:39:11.945769 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.945710 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:39:11.946819 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.946802 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:39:11.946922 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.946839 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:39:11.946922 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.946855 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:39:11.948129 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.948113 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:39:11.948258 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.948240 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.948337 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.948272 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:39:11.948931 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.948906 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:39:11.949012 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.948938 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:39:11.949012 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.948949 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:39:11.949012 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.948914 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:39:11.949180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.949018 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:39:11.949180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.949036 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:39:11.950108 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.950094 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.950161 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.950128 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:39:11.950905 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.950890 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:39:11.950958 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.950922 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:39:11.950958 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.950937 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:39:11.957005 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.956978 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.957064 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.957013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c857f80ddb13de0ede08c410a7a78ba-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"1c857f80ddb13de0ede08c410a7a78ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.957064 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:11.957032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.975283 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.975255 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:11.975366 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.975333 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-217.ec2.internal\" not found" node="ip-10-0-129-217.ec2.internal" Jul 08 17:39:11.980060 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:11.980045 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-217.ec2.internal\" not found" node="ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.057733 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.057706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c857f80ddb13de0ede08c410a7a78ba-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"1c857f80ddb13de0ede08c410a7a78ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.057733 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.057735 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.057949 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.057755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.057949 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.057813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.057949 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.057813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c857f80ddb13de0ede08c410a7a78ba-config\") pod \"kube-apiserver-proxy-ip-10-0-129-217.ec2.internal\" (UID: \"1c857f80ddb13de0ede08c410a7a78ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.057949 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.057843 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61dad3d90ea1a5450d51ca0838b0d656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal\" (UID: \"61dad3d90ea1a5450d51ca0838b0d656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.075713 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.075677 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.176741 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.176651 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.236253 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.236214 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 17:39:12.236438 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.236418 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:39:12.236484 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.236441 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:39:12.277755 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.277715 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.280051 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.280032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.284688 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.284492 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.353400 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.353344 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 17:34:11 +0000 UTC" deadline="2028-01-26 21:11:19.891676819 +0000 UTC" Jul 08 17:39:12.353400 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.353390 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13611h32m7.538290643s" Jul 08 17:39:12.378031 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.378003 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.478777 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.478713 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.579919 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.579879 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.681015 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.680979 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.782091 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.782056 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.838673 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:12.838636 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c857f80ddb13de0ede08c410a7a78ba.slice/crio-7c2a904f412656e9c3c98a98c1fbdd6f62353dd34d85b72d75f8875e05bc59d2 WatchSource:0}: Error finding container 7c2a904f412656e9c3c98a98c1fbdd6f62353dd34d85b72d75f8875e05bc59d2: Status 404 returned error can't find the container with id 7c2a904f412656e9c3c98a98c1fbdd6f62353dd34d85b72d75f8875e05bc59d2 Jul 08 17:39:12.843460 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.843418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" event={"ID":"1c857f80ddb13de0ede08c410a7a78ba","Type":"ContainerStarted","Data":"7c2a904f412656e9c3c98a98c1fbdd6f62353dd34d85b72d75f8875e05bc59d2"} Jul 08 17:39:12.843634 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.843602 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:39:12.858521 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.858499 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:39:12.882513 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:12.882474 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-217.ec2.internal\" not found" Jul 08 17:39:12.916726 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.916700 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:39:12.929723 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:12.929699 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dad3d90ea1a5450d51ca0838b0d656.slice/crio-0f7311cd2be157bda397b19a508da8464702db20a0aa1f50d06d778b70edb077 WatchSource:0}: Error finding container 0f7311cd2be157bda397b19a508da8464702db20a0aa1f50d06d778b70edb077: Status 404 returned error can't find the container with id 0f7311cd2be157bda397b19a508da8464702db20a0aa1f50d06d778b70edb077 Jul 08 17:39:12.953305 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.953278 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.966349 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.966318 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 17:39:12.967321 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.967305 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" Jul 08 17:39:12.977702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:12.977685 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 17:39:13.291670 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.291634 2580 apiserver.go:52] "Watching apiserver" Jul 08 17:39:13.297444 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.297426 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 17:39:13.300548 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.300056 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-j5t5b","openshift-network-diagnostics/network-check-target-ln4fx","kube-system/konnectivity-agent-slbgv","kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns","openshift-multus/multus-additional-cni-plugins-6w89d","openshift-multus/network-metrics-daemon-q7s9l","openshift-network-operator/iptables-alerter-h77sd","openshift-ovn-kubernetes/ovnkube-node-vxkx4","kube-system/global-pull-secret-syncer-5jp9r","openshift-cluster-node-tuning-operator/tuned-gm4nq","openshift-dns/node-resolver-44b9m","openshift-image-registry/node-ca-lb6x5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal"] Jul 08 17:39:13.302313 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.302293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.302417 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.302387 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:13.303462 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.303447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.304561 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.304531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:13.305715 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.305696 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.305823 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.305781 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6tp2n\"" Jul 08 17:39:13.305894 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.305861 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 17:39:13.305894 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.305885 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 17:39:13.306526 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.306511 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 17:39:13.306924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.306904 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 17:39:13.307054 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.306926 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.307054 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.306965 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 17:39:13.307054 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.306973 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7kr5g\"" Jul 08 17:39:13.307836 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.307815 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rj2dn\"" Jul 08 17:39:13.308028 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.308010 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 17:39:13.308320 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.308302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.308646 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.308441 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:39:13.308646 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.308447 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 17:39:13.309503 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.309487 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 17:39:13.309582 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.309494 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 17:39:13.309582 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.309568 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 17:39:13.309582 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.309576 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6fk94\"" Jul 08 17:39:13.309841 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.309826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.310409 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.310389 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 17:39:13.310505 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.310433 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-25c4c\"" Jul 08 17:39:13.310748 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.310727 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 17:39:13.310829 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.310747 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 17:39:13.310829 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.310754 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 17:39:13.311052 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.311035 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 17:39:13.311439 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.311415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:13.311529 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.311505 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:13.312002 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.311982 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 17:39:13.312370 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.312354 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 17:39:13.312370 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.312368 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4zfs8\"" Jul 08 17:39:13.312507 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.312368 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 17:39:13.313006 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.312383 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 17:39:13.313006 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.312647 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 17:39:13.313006 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.312841 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 17:39:13.313163 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.313050 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:13.313163 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.313108 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:13.314710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.314694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.316167 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.316151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.316859 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.316838 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 17:39:13.317445 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.317430 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ncdjm\"" Jul 08 17:39:13.317716 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.317697 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.318299 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.318282 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:39:13.318340 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.318309 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gcscn\"" Jul 08 17:39:13.318387 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.318287 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 17:39:13.320120 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.320103 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 17:39:13.320220 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.320110 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 17:39:13.320280 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.320164 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-c2ft6\"" Jul 08 17:39:13.353577 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.353555 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 17:39:13.354550 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.354534 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 17:39:13.364882 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.364862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-system-cni-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.364976 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.364890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-cnibin\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.364976 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.364911 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-etc-selinux\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.364976 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.364930 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-kubelet\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365098 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.364975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-socket-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.365098 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.364998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-registration-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.365098 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-etc-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365098 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-node-log\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365098 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-multus-certs\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8v6h\" (UniqueName: \"kubernetes.io/projected/51eeea3f-4533-4d4f-9048-20d7262dc02d-kube-api-access-c8v6h\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.365271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-cni-multus\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs7tt\" (UniqueName: \"kubernetes.io/projected/36f86f3b-0bc1-4921-b793-771114839622-kube-api-access-vs7tt\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.365271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-tmp\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.365271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-device-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.365271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmqz4\" (UniqueName: \"kubernetes.io/projected/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-kube-api-access-xmqz4\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-cni-binary-copy\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365268 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjmf9\" (UniqueName: \"kubernetes.io/projected/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-kube-api-access-jjmf9\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c964a201-501c-4f3b-91c1-df8d714fe4bf-agent-certs\") pod \"konnectivity-agent-slbgv\" (UID: \"c964a201-501c-4f3b-91c1-df8d714fe4bf\") " pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365305 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szmgn\" (UniqueName: \"kubernetes.io/projected/f99c796c-46cd-44c7-8d8a-28219d6a0813-kube-api-access-szmgn\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-systemd\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-log-socket\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365379 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-cni-bin\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-etc-kubernetes\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365409 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-sys-fs\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-systemd-units\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-run-netns\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365478 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51eeea3f-4533-4d4f-9048-20d7262dc02d-host\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365508 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-var-lib-kubelet\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.365538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m2w5\" (UniqueName: \"kubernetes.io/projected/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-kube-api-access-5m2w5\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365559 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365608 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-os-release\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-var-lib-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365657 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-hostroot\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-daemon-config\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365692 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-cni-binary-copy\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365714 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-cni-netd\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365747 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-netns\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-conf-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365787 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/51eeea3f-4533-4d4f-9048-20d7262dc02d-serviceca\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365814 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-sys\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-host\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.365982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365843 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-tuned\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-cni-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-os-release\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365931 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-kubelet\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysconfig\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365976 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysctl-conf\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.365998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj7ns\" (UniqueName: \"kubernetes.io/projected/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-kube-api-access-hj7ns\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36f86f3b-0bc1-4921-b793-771114839622-host-slash\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366097 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-slash\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366128 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-cni-bin\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovnkube-config\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-env-overrides\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.366430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-socket-dir-parent\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-k8s-cni-cncf-io\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366268 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366286 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-ovn\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366305 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovn-node-metrics-cert\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366321 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36f86f3b-0bc1-4921-b793-771114839622-iptables-alerter-script\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-modprobe-d\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysctl-d\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366378 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c255717f-5f5f-49fd-b0b7-8b320723e799-tmp-dir\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x56l6\" (UniqueName: \"kubernetes.io/projected/c255717f-5f5f-49fd-b0b7-8b320723e799-kube-api-access-x56l6\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366414 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c255717f-5f5f-49fd-b0b7-8b320723e799-hosts-file\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366438 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c964a201-501c-4f3b-91c1-df8d714fe4bf-konnectivity-ca\") pod \"konnectivity-agent-slbgv\" (UID: \"c964a201-501c-4f3b-91c1-df8d714fe4bf\") " pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-cnibin\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.366914 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366505 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366522 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/775aa56e-6a57-4992-8079-863793659229-kubelet-config\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-kubernetes\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-systemd\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-run\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-system-cni-dir\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovnkube-script-lib\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dznq8\" (UniqueName: \"kubernetes.io/projected/820ed21e-c804-434f-9286-3c201a9b1528-kube-api-access-dznq8\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366728 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/775aa56e-6a57-4992-8079-863793659229-dbus\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-lib-modules\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.367373 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.366774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.391733 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.391706 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vlkn4" Jul 08 17:39:13.401154 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.401137 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vlkn4" Jul 08 17:39:13.467307 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467282 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-ovn\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.467307 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovn-node-metrics-cert\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36f86f3b-0bc1-4921-b793-771114839622-iptables-alerter-script\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-modprobe-d\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467371 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysctl-d\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467388 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c255717f-5f5f-49fd-b0b7-8b320723e799-tmp-dir\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x56l6\" (UniqueName: \"kubernetes.io/projected/c255717f-5f5f-49fd-b0b7-8b320723e799-kube-api-access-x56l6\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467420 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-ovn\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c255717f-5f5f-49fd-b0b7-8b320723e799-hosts-file\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c964a201-501c-4f3b-91c1-df8d714fe4bf-konnectivity-ca\") pod \"konnectivity-agent-slbgv\" (UID: \"c964a201-501c-4f3b-91c1-df8d714fe4bf\") " pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-cnibin\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467521 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-modprobe-d\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467542 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysctl-d\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467542 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c255717f-5f5f-49fd-b0b7-8b320723e799-hosts-file\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467573 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-cnibin\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.467753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467582 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/775aa56e-6a57-4992-8079-863793659229-kubelet-config\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-kubernetes\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-systemd\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-run\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-system-cni-dir\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467789 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467789 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467832 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/775aa56e-6a57-4992-8079-863793659229-kubelet-config\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467844 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c255717f-5f5f-49fd-b0b7-8b320723e799-tmp-dir\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467886 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-systemd\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467886 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-system-cni-dir\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467894 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-run\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovnkube-script-lib\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.468489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467790 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-kubernetes\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467936 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467965 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dznq8\" (UniqueName: \"kubernetes.io/projected/820ed21e-c804-434f-9286-3c201a9b1528-kube-api-access-dznq8\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.467989 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/775aa56e-6a57-4992-8079-863793659229-dbus\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-lib-modules\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-system-cni-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36f86f3b-0bc1-4921-b793-771114839622-iptables-alerter-script\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-cnibin\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c964a201-501c-4f3b-91c1-df8d714fe4bf-konnectivity-ca\") pod \"konnectivity-agent-slbgv\" (UID: \"c964a201-501c-4f3b-91c1-df8d714fe4bf\") " pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-etc-selinux\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-kubelet\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-socket-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-registration-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.468212 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:13.469254 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468257 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-registration-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468265 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/775aa56e-6a57-4992-8079-863793659229-dbus\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.468277 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs podName:820ed21e-c804-434f-9286-3c201a9b1528 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:13.968256839 +0000 UTC m=+3.031779670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs") pod "network-metrics-daemon-q7s9l" (UID: "820ed21e-c804-434f-9286-3c201a9b1528") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468307 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-lib-modules\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468324 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-system-cni-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468365 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-kubelet\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468367 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-cnibin\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468401 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-etc-selinux\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-etc-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-etc-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468446 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-node-log\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468483 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-node-log\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468505 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-multus-certs\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.470063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-socket-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468534 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8v6h\" (UniqueName: \"kubernetes.io/projected/51eeea3f-4533-4d4f-9048-20d7262dc02d-kube-api-access-c8v6h\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468543 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-multus-certs\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468563 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-cni-multus\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468578 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs7tt\" (UniqueName: \"kubernetes.io/projected/36f86f3b-0bc1-4921-b793-771114839622-kube-api-access-vs7tt\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-tmp\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468624 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-cni-multus\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-device-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmqz4\" (UniqueName: \"kubernetes.io/projected/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-kube-api-access-xmqz4\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-cni-binary-copy\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468735 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjmf9\" (UniqueName: \"kubernetes.io/projected/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-kube-api-access-jjmf9\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468746 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-device-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c964a201-501c-4f3b-91c1-df8d714fe4bf-agent-certs\") pod \"konnectivity-agent-slbgv\" (UID: \"c964a201-501c-4f3b-91c1-df8d714fe4bf\") " pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:13.470843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szmgn\" (UniqueName: \"kubernetes.io/projected/f99c796c-46cd-44c7-8d8a-28219d6a0813-kube-api-access-szmgn\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-systemd\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-log-socket\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-cni-bin\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468898 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-systemd\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.468962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovnkube-script-lib\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-etc-kubernetes\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-sys-fs\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-systemd-units\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469105 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-run-netns\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469127 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-etc-kubernetes\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469186 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-sys-fs\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.469209 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469229 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-systemd-units\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.471592 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.469270 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret podName:775aa56e-6a57-4992-8079-863793659229 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:13.969252566 +0000 UTC m=+3.032775391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret") pod "global-pull-secret-syncer-5jp9r" (UID: "775aa56e-6a57-4992-8079-863793659229") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469269 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-run-netns\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469279 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-cni-bin\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-log-socket\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51eeea3f-4533-4d4f-9048-20d7262dc02d-host\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-var-lib-kubelet\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51eeea3f-4533-4d4f-9048-20d7262dc02d-host\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m2w5\" (UniqueName: \"kubernetes.io/projected/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-kube-api-access-5m2w5\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469442 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-var-lib-kubelet\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469456 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-os-release\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469472 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-cni-binary-copy\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-var-lib-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-hostroot\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.472166 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469526 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469534 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-var-lib-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-daemon-config\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469573 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-hostroot\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-cni-binary-copy\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-cni-netd\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-netns\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469732 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-conf-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/51eeea3f-4533-4d4f-9048-20d7262dc02d-serviceca\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-sys\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469606 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f99c796c-46cd-44c7-8d8a-28219d6a0813-os-release\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-host\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-tuned\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-cni-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.472678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469901 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-os-release\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-kubelet\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-sys\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.469993 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysconfig\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysctl-conf\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470043 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-netns\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj7ns\" (UniqueName: \"kubernetes.io/projected/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-kube-api-access-hj7ns\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470121 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-cni-binary-copy\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470134 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-conf-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36f86f3b-0bc1-4921-b793-771114839622-host-slash\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470091 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-daemon-config\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470182 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36f86f3b-0bc1-4921-b793-771114839622-host-slash\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-slash\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-cni-bin\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovnkube-config\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470310 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysconfig\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470311 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-cni-netd\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-env-overrides\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470250 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-os-release\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470379 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-cni-bin\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470402 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-host\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470453 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-var-lib-kubelet\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470486 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-host-slash\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-socket-dir-parent\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-k8s-cni-cncf-io\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470757 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-env-overrides\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.473569 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470745 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-run-openvswitch\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470805 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-host-run-k8s-cni-cncf-io\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470828 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-cni-dir\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470844 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/51eeea3f-4533-4d4f-9048-20d7262dc02d-serviceca\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470905 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-multus-socket-dir-parent\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470985 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovnkube-config\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.470992 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-sysctl-conf\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.471274 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f99c796c-46cd-44c7-8d8a-28219d6a0813-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.471963 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-ovn-node-metrics-cert\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.471982 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-tmp\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.472209 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c964a201-501c-4f3b-91c1-df8d714fe4bf-agent-certs\") pod \"konnectivity-agent-slbgv\" (UID: \"c964a201-501c-4f3b-91c1-df8d714fe4bf\") " pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:13.474116 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.472821 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-etc-tuned\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.482238 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.482203 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:39:13.482366 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.482256 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:39:13.482366 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.482273 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5dl94 for pod openshift-network-diagnostics/network-check-target-ln4fx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:13.482366 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.482358 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94 podName:ffc7691f-a570-41ef-b06e-a46d6bf50da1 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:13.982336887 +0000 UTC m=+3.045859702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5dl94" (UniqueName: "kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94") pod "network-check-target-ln4fx" (UID: "ffc7691f-a570-41ef-b06e-a46d6bf50da1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:13.482538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.482436 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dznq8\" (UniqueName: \"kubernetes.io/projected/820ed21e-c804-434f-9286-3c201a9b1528-kube-api-access-dznq8\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:13.482925 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.482897 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs7tt\" (UniqueName: \"kubernetes.io/projected/36f86f3b-0bc1-4921-b793-771114839622-kube-api-access-vs7tt\") pod \"iptables-alerter-h77sd\" (UID: \"36f86f3b-0bc1-4921-b793-771114839622\") " pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.482925 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.482917 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x56l6\" (UniqueName: \"kubernetes.io/projected/c255717f-5f5f-49fd-b0b7-8b320723e799-kube-api-access-x56l6\") pod \"node-resolver-44b9m\" (UID: \"c255717f-5f5f-49fd-b0b7-8b320723e799\") " pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.483158 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.483129 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmqz4\" (UniqueName: \"kubernetes.io/projected/47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf-kube-api-access-xmqz4\") pod \"ovnkube-node-vxkx4\" (UID: \"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.483800 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.483772 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8v6h\" (UniqueName: \"kubernetes.io/projected/51eeea3f-4533-4d4f-9048-20d7262dc02d-kube-api-access-c8v6h\") pod \"node-ca-lb6x5\" (UID: \"51eeea3f-4533-4d4f-9048-20d7262dc02d\") " pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.485533 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.485167 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szmgn\" (UniqueName: \"kubernetes.io/projected/f99c796c-46cd-44c7-8d8a-28219d6a0813-kube-api-access-szmgn\") pod \"multus-additional-cni-plugins-6w89d\" (UID: \"f99c796c-46cd-44c7-8d8a-28219d6a0813\") " pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.485659 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.485544 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m2w5\" (UniqueName: \"kubernetes.io/projected/2b83bf25-81fa-4a60-b5f9-1e0467c378b7-kube-api-access-5m2w5\") pod \"tuned-gm4nq\" (UID: \"2b83bf25-81fa-4a60-b5f9-1e0467c378b7\") " pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.486583 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.486550 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj7ns\" (UniqueName: \"kubernetes.io/projected/10164a07-9f06-4ebb-8d1e-8d47ba6b4582-kube-api-access-hj7ns\") pod \"aws-ebs-csi-driver-node-wrwns\" (UID: \"10164a07-9f06-4ebb-8d1e-8d47ba6b4582\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.486743 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.486718 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjmf9\" (UniqueName: \"kubernetes.io/projected/3c26000d-c01e-48ed-8cb7-03b67b9cfb28-kube-api-access-jjmf9\") pod \"multus-j5t5b\" (UID: \"3c26000d-c01e-48ed-8cb7-03b67b9cfb28\") " pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.520518 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.520492 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:39:13.571468 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.571400 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:39:13.616284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.616215 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lb6x5" Jul 08 17:39:13.622505 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.622476 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:13.627543 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:13.627510 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51eeea3f_4533_4d4f_9048_20d7262dc02d.slice/crio-3d69d23e97e46295c35c126c85d9df9a21272d63bbbdaf701ce8f06b939b9871 WatchSource:0}: Error finding container 3d69d23e97e46295c35c126c85d9df9a21272d63bbbdaf701ce8f06b939b9871: Status 404 returned error can't find the container with id 3d69d23e97e46295c35c126c85d9df9a21272d63bbbdaf701ce8f06b939b9871 Jul 08 17:39:13.630353 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.630327 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h77sd" Jul 08 17:39:13.634845 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:13.634815 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc964a201_501c_4f3b_91c1_df8d714fe4bf.slice/crio-6367b45f4bd8ba43e4b3e42a8bdf282af76c64d8f7753c0d4a5f62ceba53a4c5 WatchSource:0}: Error finding container 6367b45f4bd8ba43e4b3e42a8bdf282af76c64d8f7753c0d4a5f62ceba53a4c5: Status 404 returned error can't find the container with id 6367b45f4bd8ba43e4b3e42a8bdf282af76c64d8f7753c0d4a5f62ceba53a4c5 Jul 08 17:39:13.636050 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.636029 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" Jul 08 17:39:13.641413 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.640914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6w89d" Jul 08 17:39:13.645191 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:13.645156 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f86f3b_0bc1_4921_b793_771114839622.slice/crio-3571af4ebbc7c1f0e584ae62a3a4f8b14b885d4704a4b8fba0769c6efb60d9a2 WatchSource:0}: Error finding container 3571af4ebbc7c1f0e584ae62a3a4f8b14b885d4704a4b8fba0769c6efb60d9a2: Status 404 returned error can't find the container with id 3571af4ebbc7c1f0e584ae62a3a4f8b14b885d4704a4b8fba0769c6efb60d9a2 Jul 08 17:39:13.647470 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.647301 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:13.652794 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:13.652764 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10164a07_9f06_4ebb_8d1e_8d47ba6b4582.slice/crio-7b77ae90dbae1c3f00024eafbb064db62d9670dba52d31aab8eb93925dc99d62 WatchSource:0}: Error finding container 7b77ae90dbae1c3f00024eafbb064db62d9670dba52d31aab8eb93925dc99d62: Status 404 returned error can't find the container with id 7b77ae90dbae1c3f00024eafbb064db62d9670dba52d31aab8eb93925dc99d62 Jul 08 17:39:13.654050 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.654024 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j5t5b" Jul 08 17:39:13.657116 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:13.657059 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99c796c_46cd_44c7_8d8a_28219d6a0813.slice/crio-40d2cc72873407eaeab41167ddabaeb7bbe604854a81e10e8e8db7662c895c4a WatchSource:0}: Error finding container 40d2cc72873407eaeab41167ddabaeb7bbe604854a81e10e8e8db7662c895c4a: Status 404 returned error can't find the container with id 40d2cc72873407eaeab41167ddabaeb7bbe604854a81e10e8e8db7662c895c4a Jul 08 17:39:13.658981 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.658949 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" Jul 08 17:39:13.665369 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:13.665270 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d1ea9b_26f9_45a7_ab6f_9920b2f71eaf.slice/crio-d4ff58e14e511ab93067963161f412fc0acbcb15adf1ce1a23ef48f4f9f719a7 WatchSource:0}: Error finding container d4ff58e14e511ab93067963161f412fc0acbcb15adf1ce1a23ef48f4f9f719a7: Status 404 returned error can't find the container with id d4ff58e14e511ab93067963161f412fc0acbcb15adf1ce1a23ef48f4f9f719a7 Jul 08 17:39:13.665369 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.665364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-44b9m" Jul 08 17:39:13.681636 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:13.681604 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c26000d_c01e_48ed_8cb7_03b67b9cfb28.slice/crio-5a8f2bde51ce33acfdfc4db91e37cf369e141aad6edadd2360a42ebe0e33025c WatchSource:0}: Error finding container 5a8f2bde51ce33acfdfc4db91e37cf369e141aad6edadd2360a42ebe0e33025c: Status 404 returned error can't find the container with id 5a8f2bde51ce33acfdfc4db91e37cf369e141aad6edadd2360a42ebe0e33025c Jul 08 17:39:13.684836 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:39:13.684799 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc255717f_5f5f_49fd_b0b7_8b320723e799.slice/crio-2fc8b121c0545382950e960e5fc10e9321572bc17961bc2f2ccc4ffd0529fc62 WatchSource:0}: Error finding container 2fc8b121c0545382950e960e5fc10e9321572bc17961bc2f2ccc4ffd0529fc62: Status 404 returned error can't find the container with id 2fc8b121c0545382950e960e5fc10e9321572bc17961bc2f2ccc4ffd0529fc62 Jul 08 17:39:13.847433 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.847343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-44b9m" event={"ID":"c255717f-5f5f-49fd-b0b7-8b320723e799","Type":"ContainerStarted","Data":"2fc8b121c0545382950e960e5fc10e9321572bc17961bc2f2ccc4ffd0529fc62"} Jul 08 17:39:13.849635 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.849576 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" event={"ID":"2b83bf25-81fa-4a60-b5f9-1e0467c378b7","Type":"ContainerStarted","Data":"cbfe4e456b112c37789caf963f7544730e4b6b4e4f3d935493c04c0eaa587009"} Jul 08 17:39:13.851337 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.851310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j5t5b" event={"ID":"3c26000d-c01e-48ed-8cb7-03b67b9cfb28","Type":"ContainerStarted","Data":"5a8f2bde51ce33acfdfc4db91e37cf369e141aad6edadd2360a42ebe0e33025c"} Jul 08 17:39:13.853033 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.853005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" event={"ID":"10164a07-9f06-4ebb-8d1e-8d47ba6b4582","Type":"ContainerStarted","Data":"7b77ae90dbae1c3f00024eafbb064db62d9670dba52d31aab8eb93925dc99d62"} Jul 08 17:39:13.854637 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.854612 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h77sd" event={"ID":"36f86f3b-0bc1-4921-b793-771114839622","Type":"ContainerStarted","Data":"3571af4ebbc7c1f0e584ae62a3a4f8b14b885d4704a4b8fba0769c6efb60d9a2"} Jul 08 17:39:13.855938 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.855905 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"61dad3d90ea1a5450d51ca0838b0d656","Type":"ContainerStarted","Data":"0f7311cd2be157bda397b19a508da8464702db20a0aa1f50d06d778b70edb077"} Jul 08 17:39:13.859175 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.859146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"d4ff58e14e511ab93067963161f412fc0acbcb15adf1ce1a23ef48f4f9f719a7"} Jul 08 17:39:13.861307 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.861270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6w89d" event={"ID":"f99c796c-46cd-44c7-8d8a-28219d6a0813","Type":"ContainerStarted","Data":"40d2cc72873407eaeab41167ddabaeb7bbe604854a81e10e8e8db7662c895c4a"} Jul 08 17:39:13.863747 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.863716 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-slbgv" event={"ID":"c964a201-501c-4f3b-91c1-df8d714fe4bf","Type":"ContainerStarted","Data":"6367b45f4bd8ba43e4b3e42a8bdf282af76c64d8f7753c0d4a5f62ceba53a4c5"} Jul 08 17:39:13.865242 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.865218 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lb6x5" event={"ID":"51eeea3f-4533-4d4f-9048-20d7262dc02d","Type":"ContainerStarted","Data":"3d69d23e97e46295c35c126c85d9df9a21272d63bbbdaf701ce8f06b939b9871"} Jul 08 17:39:13.974742 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.974704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:13.974951 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:13.974778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:13.974951 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.974903 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:13.974951 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.974904 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:13.975120 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.974982 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs podName:820ed21e-c804-434f-9286-3c201a9b1528 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:14.974959934 +0000 UTC m=+4.038482743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs") pod "network-metrics-daemon-q7s9l" (UID: "820ed21e-c804-434f-9286-3c201a9b1528") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:13.975120 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:13.975002 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret podName:775aa56e-6a57-4992-8079-863793659229 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:14.974992307 +0000 UTC m=+4.038515123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret") pod "global-pull-secret-syncer-5jp9r" (UID: "775aa56e-6a57-4992-8079-863793659229") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:14.075797 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:14.075755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:14.076063 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.076028 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:39:14.076063 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.076065 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:39:14.076233 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.076081 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5dl94 for pod openshift-network-diagnostics/network-check-target-ln4fx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:14.076233 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.076156 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94 podName:ffc7691f-a570-41ef-b06e-a46d6bf50da1 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:15.076134909 +0000 UTC m=+4.139657720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dl94" (UniqueName: "kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94") pod "network-check-target-ln4fx" (UID: "ffc7691f-a570-41ef-b06e-a46d6bf50da1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:14.403840 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:14.403793 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:34:13 +0000 UTC" deadline="2028-02-18 07:54:31.461146791 +0000 UTC" Jul 08 17:39:14.403840 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:14.403838 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14150h15m17.057313619s" Jul 08 17:39:14.841017 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:14.840937 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:14.841166 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.841074 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:14.841350 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:14.841327 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:14.841470 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.841446 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:14.841578 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:14.841532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:14.841692 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.841669 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:14.984192 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:14.984152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:14.984380 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:14.984236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:14.984451 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.984410 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:14.984510 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.984477 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret podName:775aa56e-6a57-4992-8079-863793659229 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:16.984456773 +0000 UTC m=+6.047979587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret") pod "global-pull-secret-syncer-5jp9r" (UID: "775aa56e-6a57-4992-8079-863793659229") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:14.984950 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.984930 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:14.985047 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:14.984987 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs podName:820ed21e-c804-434f-9286-3c201a9b1528 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:16.984970166 +0000 UTC m=+6.048492978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs") pod "network-metrics-daemon-q7s9l" (UID: "820ed21e-c804-434f-9286-3c201a9b1528") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:15.085782 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:15.085712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:15.085988 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:15.085938 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:39:15.085988 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:15.085967 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:39:15.085988 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:15.085982 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5dl94 for pod openshift-network-diagnostics/network-check-target-ln4fx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:15.086150 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:15.086043 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94 podName:ffc7691f-a570-41ef-b06e-a46d6bf50da1 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:17.086023241 +0000 UTC m=+6.149546070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dl94" (UniqueName: "kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94") pod "network-check-target-ln4fx" (UID: "ffc7691f-a570-41ef-b06e-a46d6bf50da1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:15.405245 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:15.405169 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:34:13 +0000 UTC" deadline="2028-03-06 07:25:31.509505219 +0000 UTC" Jul 08 17:39:15.405245 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:15.405243 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14557h46m16.104265558s" Jul 08 17:39:16.840587 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:16.840550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:16.841243 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:16.840716 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:16.841243 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:16.841152 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:16.841392 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:16.841264 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:16.841392 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:16.841325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:16.841498 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:16.841413 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:17.001017 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:17.000969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:17.001197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:17.001051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:17.001197 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:17.001129 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:17.001197 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:17.001164 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:17.001357 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:17.001205 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs podName:820ed21e-c804-434f-9286-3c201a9b1528 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:21.001183802 +0000 UTC m=+10.064706650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs") pod "network-metrics-daemon-q7s9l" (UID: "820ed21e-c804-434f-9286-3c201a9b1528") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:17.001357 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:17.001225 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret podName:775aa56e-6a57-4992-8079-863793659229 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:21.001214693 +0000 UTC m=+10.064737510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret") pod "global-pull-secret-syncer-5jp9r" (UID: "775aa56e-6a57-4992-8079-863793659229") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:17.101550 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:17.101462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:17.101766 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:17.101657 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:39:17.101766 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:17.101677 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:39:17.101766 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:17.101691 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5dl94 for pod openshift-network-diagnostics/network-check-target-ln4fx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:17.101766 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:17.101751 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94 podName:ffc7691f-a570-41ef-b06e-a46d6bf50da1 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:21.101732598 +0000 UTC m=+10.165255409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dl94" (UniqueName: "kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94") pod "network-check-target-ln4fx" (UID: "ffc7691f-a570-41ef-b06e-a46d6bf50da1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:18.841153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:18.840362 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:18.841153 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:18.840503 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:18.841153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:18.840924 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:18.841153 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:18.841038 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:18.841153 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:18.841140 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:18.841836 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:18.841255 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:20.841740 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:20.841202 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:20.841740 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:20.841202 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:20.841740 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:20.841343 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:20.841740 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:20.841209 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:20.841740 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:20.841416 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:20.841740 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:20.841610 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:21.035060 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:21.034977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:21.035212 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:21.035061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:21.035212 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:21.035186 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:21.035328 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:21.035245 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs podName:820ed21e-c804-434f-9286-3c201a9b1528 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:29.035227221 +0000 UTC m=+18.098750033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs") pod "network-metrics-daemon-q7s9l" (UID: "820ed21e-c804-434f-9286-3c201a9b1528") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:21.035583 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:21.035551 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:21.035708 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:21.035642 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret podName:775aa56e-6a57-4992-8079-863793659229 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:29.035623622 +0000 UTC m=+18.099146446 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret") pod "global-pull-secret-syncer-5jp9r" (UID: "775aa56e-6a57-4992-8079-863793659229") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:21.136426 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:21.136382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:21.136626 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:21.136593 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:39:21.136626 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:21.136621 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:39:21.136744 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:21.136636 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5dl94 for pod openshift-network-diagnostics/network-check-target-ln4fx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:21.136744 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:21.136695 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94 podName:ffc7691f-a570-41ef-b06e-a46d6bf50da1 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:29.136675185 +0000 UTC m=+18.200197991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dl94" (UniqueName: "kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94") pod "network-check-target-ln4fx" (UID: "ffc7691f-a570-41ef-b06e-a46d6bf50da1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:22.840614 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:22.840558 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:22.840614 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:22.840591 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:22.841107 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:22.840705 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:22.841107 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:22.840714 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:22.841107 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:22.840846 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:22.841107 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:22.840925 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:24.840300 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:24.840262 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:24.840720 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:24.840385 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:24.840720 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:24.840396 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:24.840720 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:24.840409 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:24.840720 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:24.840498 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:24.840720 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:24.840597 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:26.841064 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:26.841032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:26.841509 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:26.841112 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:26.841509 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:26.841230 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:26.841509 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:26.841239 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:26.841509 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:26.841358 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:26.841509 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:26.841441 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:28.840893 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:28.840861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:28.840893 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:28.840895 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:28.841429 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:28.840909 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:28.841429 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:28.840989 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:28.841429 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:28.841091 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:28.841564 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:28.841499 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:29.092169 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:29.092102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:29.092169 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:29.092148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:29.092346 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:29.092278 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:29.092346 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:29.092286 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:29.092346 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:29.092337 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret podName:775aa56e-6a57-4992-8079-863793659229 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:45.092320412 +0000 UTC m=+34.155843221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret") pod "global-pull-secret-syncer-5jp9r" (UID: "775aa56e-6a57-4992-8079-863793659229") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:29.092450 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:29.092350 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs podName:820ed21e-c804-434f-9286-3c201a9b1528 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:45.092344675 +0000 UTC m=+34.155867482 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs") pod "network-metrics-daemon-q7s9l" (UID: "820ed21e-c804-434f-9286-3c201a9b1528") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:29.193002 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:29.192964 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:29.193163 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:29.193132 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:39:29.193227 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:29.193167 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:39:29.193227 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:29.193182 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5dl94 for pod openshift-network-diagnostics/network-check-target-ln4fx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:29.193315 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:29.193237 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94 podName:ffc7691f-a570-41ef-b06e-a46d6bf50da1 nodeName:}" failed. No retries permitted until 2026-07-08 17:39:45.19322401 +0000 UTC m=+34.256746822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dl94" (UniqueName: "kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94") pod "network-check-target-ln4fx" (UID: "ffc7691f-a570-41ef-b06e-a46d6bf50da1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:30.840800 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:30.840761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:30.840800 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:30.840788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:30.841312 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:30.840804 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:30.841312 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:30.840894 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:30.841312 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:30.841033 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:30.841312 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:30.841116 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:32.840308 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:32.840273 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:32.840712 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:32.840312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:32.840712 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:32.840312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:32.840712 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:32.840400 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:32.840712 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:32.840496 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:32.840712 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:32.840588 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:33.915689 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:33.915404 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"1484b4669433102de212630a071af54e780581eda8f81caa8f483e51bac2cceb"} Jul 08 17:39:33.920343 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:33.920311 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" event={"ID":"1c857f80ddb13de0ede08c410a7a78ba","Type":"ContainerStarted","Data":"09e8ea201a0c8e6be477fb4e29f20d8f924f8f31888b33730a4f399d15089a35"} Jul 08 17:39:33.925345 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:33.925322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" event={"ID":"2b83bf25-81fa-4a60-b5f9-1e0467c378b7","Type":"ContainerStarted","Data":"0023ec465b9d7aec3fe84fa2f5230fa597eda12b9dcf6b5e37c78a62a6ff5a3f"} Jul 08 17:39:33.930222 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:33.929894 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j5t5b" event={"ID":"3c26000d-c01e-48ed-8cb7-03b67b9cfb28","Type":"ContainerStarted","Data":"c4d2032dfb3b95fe5028b1867f926c8756b73709f1a6b4327aaac80eaa96c63b"} Jul 08 17:39:33.936809 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:33.936757 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-217.ec2.internal" podStartSLOduration=20.936741276 podStartE2EDuration="20.936741276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:33.936528424 +0000 UTC m=+23.000051253" watchObservedRunningTime="2026-07-08 17:39:33.936741276 +0000 UTC m=+23.000264108" Jul 08 17:39:33.972457 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:33.972288 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gm4nq" podStartSLOduration=2.196623103 podStartE2EDuration="21.972272789s" totalImagesPullingTime="19.775649686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:33.97184652 +0000 UTC m=+23.035369353" watchObservedRunningTime="2026-07-08 17:39:33.972272789 +0000 UTC m=+23.035795664" Jul 08 17:39:33.972681 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:33.972616 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-j5t5b" podStartSLOduration=2.165064129 podStartE2EDuration="21.972610698s" totalImagesPullingTime="19.807546569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:33.954025848 +0000 UTC m=+23.017548678" watchObservedRunningTime="2026-07-08 17:39:33.972610698 +0000 UTC m=+23.036133526" Jul 08 17:39:34.841079 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.841048 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:34.841079 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.841073 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:34.841308 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:34.841156 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:34.841308 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.841213 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:34.841417 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:34.841340 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:34.841470 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:34.841441 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:34.935161 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.935083 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"11fb18acc41a803a09ca280a957d1d5792c896dff4ed4bc18f668099966762b1"} Jul 08 17:39:34.935901 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.935169 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"ad59d0946e82e9936b5862e85f2bf0f37d395cba9f3a39f1402c98b3e853fa30"} Jul 08 17:39:34.935901 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.935213 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"989bf1bbb5e4eda17b25a84d3fcfc5c918b44f13aec14c56e04adbab7be7e00e"} Jul 08 17:39:34.935901 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.935254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"c50b5845bab9d3513a34e89389737741247b253fa9e780a6884c21466a1ca7d7"} Jul 08 17:39:34.935901 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.935294 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"691dd40754c344936b9532b56cb13802244690d10c0eccbf044ab2a7d0caec53"} Jul 08 17:39:34.936563 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.936538 2580 generic.go:356] "Generic (PLEG): container finished" podID="f99c796c-46cd-44c7-8d8a-28219d6a0813" containerID="682c50f11f129557206bbcaa22940aab8b03dac5c574fbaf3c779b1de29e9002" exitCode=0 Jul 08 17:39:34.936645 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.936626 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6w89d" event={"ID":"f99c796c-46cd-44c7-8d8a-28219d6a0813","Type":"ContainerDied","Data":"682c50f11f129557206bbcaa22940aab8b03dac5c574fbaf3c779b1de29e9002"} Jul 08 17:39:34.938177 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.938131 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-slbgv" event={"ID":"c964a201-501c-4f3b-91c1-df8d714fe4bf","Type":"ContainerStarted","Data":"5f7677cb3de60387960b8a249bb17b1c4271eb9f7e5c237b61619cb72ca0d9e3"} Jul 08 17:39:34.939814 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.939792 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lb6x5" event={"ID":"51eeea3f-4533-4d4f-9048-20d7262dc02d","Type":"ContainerStarted","Data":"c93d9ac5747213715a17070728d9c347a5c7adc2dbd92e7ba06c0813edd45859"} Jul 08 17:39:34.941239 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.941216 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-44b9m" event={"ID":"c255717f-5f5f-49fd-b0b7-8b320723e799","Type":"ContainerStarted","Data":"08ad0b547a685bd9d51a4f572b50e25cade9c24e3bad2ceee76326ca537b83c9"} Jul 08 17:39:34.942678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.942659 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" event={"ID":"10164a07-9f06-4ebb-8d1e-8d47ba6b4582","Type":"ContainerStarted","Data":"4f8eb4d64a5bfee7a78656366f10d80b740255feb022f24151b961bca974aa91"} Jul 08 17:39:34.944044 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.944021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h77sd" event={"ID":"36f86f3b-0bc1-4921-b793-771114839622","Type":"ContainerStarted","Data":"5007006cdf345a09c6b820b5217da2e1f1ae89e73fd488a7e3fce8e24f2c45c9"} Jul 08 17:39:34.945333 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.945315 2580 generic.go:356] "Generic (PLEG): container finished" podID="61dad3d90ea1a5450d51ca0838b0d656" containerID="b9b805828826810bd58fbe7d1d063a6d889ecbe9671c5e5b5fe2bf05c59fd305" exitCode=0 Jul 08 17:39:34.945425 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.945405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"61dad3d90ea1a5450d51ca0838b0d656","Type":"ContainerDied","Data":"b9b805828826810bd58fbe7d1d063a6d889ecbe9671c5e5b5fe2bf05c59fd305"} Jul 08 17:39:34.973080 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.973040 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lb6x5" podStartSLOduration=3.410902018 podStartE2EDuration="22.973026997s" totalImagesPullingTime="19.562124979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:34.972737208 +0000 UTC m=+24.036260037" watchObservedRunningTime="2026-07-08 17:39:34.973026997 +0000 UTC m=+24.036549826" Jul 08 17:39:34.988025 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:34.987976 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-slbgv" podStartSLOduration=11.503494444 podStartE2EDuration="22.987960855s" totalImagesPullingTime="11.484466411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:34.987753284 +0000 UTC m=+24.051276113" watchObservedRunningTime="2026-07-08 17:39:34.987960855 +0000 UTC m=+24.051483684" Jul 08 17:39:35.003767 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:35.003722 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-44b9m" podStartSLOduration=16.970477012 podStartE2EDuration="23.003711665s" totalImagesPullingTime="6.033234653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:35.003527295 +0000 UTC m=+24.067050125" watchObservedRunningTime="2026-07-08 17:39:35.003711665 +0000 UTC m=+24.067234493" Jul 08 17:39:35.049679 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:35.049641 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h77sd" podStartSLOduration=7.718495838 podStartE2EDuration="23.049621904s" totalImagesPullingTime="15.331126066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:35.049510061 +0000 UTC m=+24.113032889" watchObservedRunningTime="2026-07-08 17:39:35.049621904 +0000 UTC m=+24.113144733" Jul 08 17:39:35.953217 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:35.953183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" event={"ID":"61dad3d90ea1a5450d51ca0838b0d656","Type":"ContainerStarted","Data":"1c3d4c641f5c66b9102f3d0335d3f29198205ea52663dc5d96b740b183095ee4"} Jul 08 17:39:35.969385 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:35.969329 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-217.ec2.internal" podStartSLOduration=22.969309141 podStartE2EDuration="22.969309141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:35.968488891 +0000 UTC m=+25.032011720" watchObservedRunningTime="2026-07-08 17:39:35.969309141 +0000 UTC m=+25.032831973" Jul 08 17:39:36.048298 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.048261 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 17:39:36.451456 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.451371 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T17:39:36.048284081Z","UUID":"77923333-a007-41d4-8d9f-3a550bd13ebe","Handler":null,"Name":"","Endpoint":""} Jul 08 17:39:36.452742 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.452720 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 17:39:36.452861 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.452758 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 17:39:36.825628 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.825523 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-44b9m_c255717f-5f5f-49fd-b0b7-8b320723e799/dns-node-resolver/0.log" Jul 08 17:39:36.840824 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.840792 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:36.840992 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.840839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:36.840992 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.840839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:36.840992 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:36.840908 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:36.841154 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:36.840992 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:36.841154 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:36.841080 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:36.957730 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.957692 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" event={"ID":"10164a07-9f06-4ebb-8d1e-8d47ba6b4582","Type":"ContainerStarted","Data":"3ecca7a9a00ab75fdd942184b88258f3c8b80df8cd58ef0da80afb2a41c46727"} Jul 08 17:39:36.962070 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:36.962040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"79c1f27f3720b4cbd2e526db8c1a8aae44c9b38207acf62edc41cfb413e62c7d"} Jul 08 17:39:37.802054 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:37.802026 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lb6x5_51eeea3f-4533-4d4f-9048-20d7262dc02d/node-ca/0.log" Jul 08 17:39:37.921105 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:37.921074 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:37.921700 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:37.921676 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:37.967092 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:37.967047 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" event={"ID":"10164a07-9f06-4ebb-8d1e-8d47ba6b4582","Type":"ContainerStarted","Data":"586eff51db55ca5940fff41bb119492b16904ce7e8ed61df2fef7dad64298159"} Jul 08 17:39:37.967556 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:37.967283 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:37.968251 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:37.968227 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-slbgv" Jul 08 17:39:37.990924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:37.990877 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wrwns" podStartSLOduration=3.912042591 podStartE2EDuration="25.990861736s" totalImagesPullingTime="22.078819145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:37.99037653 +0000 UTC m=+27.053899369" watchObservedRunningTime="2026-07-08 17:39:37.990861736 +0000 UTC m=+27.054384567" Jul 08 17:39:38.841518 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:38.841310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:38.841647 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:38.841364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:38.841647 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:38.841627 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:38.841731 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:38.841364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:38.841731 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:38.841699 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:38.841799 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:38.841773 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:38.973909 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:38.973686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" event={"ID":"47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf","Type":"ContainerStarted","Data":"72326fdafd62b939e8cfdfe6cf7817d3b18d7c5fe8be725bad41b43fd394ade4"} Jul 08 17:39:39.000481 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:39.000437 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" podStartSLOduration=6.854087014 podStartE2EDuration="27.000421173s" totalImagesPullingTime="20.146334159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:38.999490521 +0000 UTC m=+28.063013350" watchObservedRunningTime="2026-07-08 17:39:39.000421173 +0000 UTC m=+28.063944024" Jul 08 17:39:39.976392 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:39.976365 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 17:39:39.976964 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:39.976814 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:39.976964 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:39.976897 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:39.994423 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:39.994391 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:39.994584 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:39.994528 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:40.841257 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:40.841215 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:40.841257 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:40.841238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:40.841502 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:40.841353 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:40.841502 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:40.841354 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:40.841502 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:40.841440 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:40.841655 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:40.841510 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:40.978930 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:40.978901 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 17:39:41.982602 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:41.982557 2580 generic.go:356] "Generic (PLEG): container finished" podID="f99c796c-46cd-44c7-8d8a-28219d6a0813" containerID="ab1944c951595055db790954db0e2b18f6750e2dac29f3df787f793879a6d079" exitCode=0 Jul 08 17:39:41.983194 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:41.982643 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6w89d" event={"ID":"f99c796c-46cd-44c7-8d8a-28219d6a0813","Type":"ContainerDied","Data":"ab1944c951595055db790954db0e2b18f6750e2dac29f3df787f793879a6d079"} Jul 08 17:39:41.983194 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:41.982701 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 17:39:42.840728 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:42.840701 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:42.840844 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:42.840820 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:42.840897 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:42.840838 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:42.840897 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:42.840869 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:42.841012 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:42.840990 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:42.841098 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:42.841078 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:42.986544 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:42.986514 2580 generic.go:356] "Generic (PLEG): container finished" podID="f99c796c-46cd-44c7-8d8a-28219d6a0813" containerID="5aaa17b2e0bc479962ea60f758ddd32265a159103aca91b90a18f2add5a1af72" exitCode=0 Jul 08 17:39:42.986877 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:42.986561 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6w89d" event={"ID":"f99c796c-46cd-44c7-8d8a-28219d6a0813","Type":"ContainerDied","Data":"5aaa17b2e0bc479962ea60f758ddd32265a159103aca91b90a18f2add5a1af72"} Jul 08 17:39:43.343785 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:43.343754 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:39:43.343972 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:43.343961 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 17:39:43.353307 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:43.353263 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" podUID="47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf" containerName="ovnkube-controller" probeResult="failure" output="" Jul 08 17:39:43.361752 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:43.361722 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" podUID="47d1ea9b-26f9-45a7-ab6f-9920b2f71eaf" containerName="ovnkube-controller" probeResult="failure" output="" Jul 08 17:39:43.990864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:43.990833 2580 generic.go:356] "Generic (PLEG): container finished" podID="f99c796c-46cd-44c7-8d8a-28219d6a0813" containerID="d3274782ead2c43c525df3cdd9460b3b7e35702b20197633243b93201708d644" exitCode=0 Jul 08 17:39:43.991191 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:43.990863 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6w89d" event={"ID":"f99c796c-46cd-44c7-8d8a-28219d6a0813","Type":"ContainerDied","Data":"d3274782ead2c43c525df3cdd9460b3b7e35702b20197633243b93201708d644"} Jul 08 17:39:44.841059 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:44.841014 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:44.841059 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:44.841047 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:44.841280 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:44.841124 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:44.841280 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:44.841131 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:44.841280 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:44.841199 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:44.841280 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:44.841261 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:45.115305 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:45.115229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:45.115305 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:45.115275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:45.115771 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:45.115385 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:45.115771 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:45.115393 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:45.115771 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:45.115433 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret podName:775aa56e-6a57-4992-8079-863793659229 nodeName:}" failed. No retries permitted until 2026-07-08 17:40:17.11542069 +0000 UTC m=+66.178943497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret") pod "global-pull-secret-syncer-5jp9r" (UID: "775aa56e-6a57-4992-8079-863793659229") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:39:45.115771 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:45.115446 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs podName:820ed21e-c804-434f-9286-3c201a9b1528 nodeName:}" failed. No retries permitted until 2026-07-08 17:40:17.115440544 +0000 UTC m=+66.178963351 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs") pod "network-metrics-daemon-q7s9l" (UID: "820ed21e-c804-434f-9286-3c201a9b1528") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:39:45.215960 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:45.215930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:45.216113 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:45.216071 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:39:45.216113 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:45.216083 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:39:45.216113 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:45.216093 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5dl94 for pod openshift-network-diagnostics/network-check-target-ln4fx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:45.216230 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:45.216136 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94 podName:ffc7691f-a570-41ef-b06e-a46d6bf50da1 nodeName:}" failed. No retries permitted until 2026-07-08 17:40:17.216124198 +0000 UTC m=+66.279647004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dl94" (UniqueName: "kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94") pod "network-check-target-ln4fx" (UID: "ffc7691f-a570-41ef-b06e-a46d6bf50da1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:39:46.841324 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:46.841291 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:46.841850 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:46.841291 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:46.841850 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:46.841415 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:46.841850 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:46.841303 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:46.841850 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:46.841473 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:46.841850 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:46.841550 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:48.840483 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:48.840449 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:48.840483 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:48.840480 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:48.841024 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:48.840480 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:48.841024 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:48.840593 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:48.841024 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:48.840657 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:48.841024 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:48.840756 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:50.005528 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:50.005455 2580 generic.go:356] "Generic (PLEG): container finished" podID="f99c796c-46cd-44c7-8d8a-28219d6a0813" containerID="1aaa80b938016e6f00d151e9d1457b55cdf5a12d35449e106fa06ffbfbfe8d87" exitCode=0 Jul 08 17:39:50.005528 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:50.005497 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6w89d" event={"ID":"f99c796c-46cd-44c7-8d8a-28219d6a0813","Type":"ContainerDied","Data":"1aaa80b938016e6f00d151e9d1457b55cdf5a12d35449e106fa06ffbfbfe8d87"} Jul 08 17:39:50.841102 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:50.840920 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:50.841265 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:50.840920 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:50.841265 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:50.841174 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:50.841265 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:50.840920 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:50.841265 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:50.841239 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:50.841406 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:50.841314 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:51.010634 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:51.010602 2580 generic.go:356] "Generic (PLEG): container finished" podID="f99c796c-46cd-44c7-8d8a-28219d6a0813" containerID="7893dbf512bc3a1950c818dcb98cff1b245b66d7ff35a29824ab0e5aa4ce0d55" exitCode=0 Jul 08 17:39:51.011028 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:51.010646 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6w89d" event={"ID":"f99c796c-46cd-44c7-8d8a-28219d6a0813","Type":"ContainerDied","Data":"7893dbf512bc3a1950c818dcb98cff1b245b66d7ff35a29824ab0e5aa4ce0d55"} Jul 08 17:39:52.015758 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:52.015725 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6w89d" event={"ID":"f99c796c-46cd-44c7-8d8a-28219d6a0813","Type":"ContainerStarted","Data":"e66798b1983699a12e92fa3f9f2d712886474cc30e6882dd3bb8e25da3ce5810"} Jul 08 17:39:52.039947 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:52.039905 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6w89d" podStartSLOduration=6.324633787 podStartE2EDuration="40.039890938s" totalImagesPullingTime="33.715257151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:52.039444383 +0000 UTC m=+41.102967212" watchObservedRunningTime="2026-07-08 17:39:52.039890938 +0000 UTC m=+41.103413793" Jul 08 17:39:52.841299 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:52.841252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:52.841470 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:52.841306 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:52.841470 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:52.841266 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:52.841470 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:52.841377 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:52.841470 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:52.841441 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:52.841624 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:52.841493 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:54.840677 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:54.840652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:54.841027 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:54.840654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:54.841027 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:54.840747 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:54.841027 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:54.840654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:54.841027 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:54.840813 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:54.841027 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:54.840879 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:55.841278 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:55.841241 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:55.841672 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:55.841367 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:56.841348 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:56.841316 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:56.841348 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:56.841352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:56.841823 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:56.841429 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:56.841823 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:56.841602 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:57.710103 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:57.710067 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5jp9r"] Jul 08 17:39:57.710255 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:57.710185 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:57.710319 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:57.710298 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:39:57.711968 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:57.711911 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ln4fx"] Jul 08 17:39:57.712091 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:57.712028 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:57.712149 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:57.712126 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:57.712691 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:57.712667 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q7s9l"] Jul 08 17:39:57.712800 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:57.712749 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:57.712857 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:57.712837 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:59.840866 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:59.840836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:39:59.841205 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:59.840960 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:39:59.841205 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:59.841035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:39:59.841205 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:59.841135 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:39:59.841323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:39:59.841211 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:39:59.844128 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:39:59.841610 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:40:01.841796 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:01.841622 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:40:01.842236 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:01.841628 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:40:01.842236 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:01.841890 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7s9l" podUID="820ed21e-c804-434f-9286-3c201a9b1528" Jul 08 17:40:01.842236 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:01.841933 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln4fx" podUID="ffc7691f-a570-41ef-b06e-a46d6bf50da1" Jul 08 17:40:01.842236 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:01.841650 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:40:01.842236 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:01.842030 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5jp9r" podUID="775aa56e-6a57-4992-8079-863793659229" Jul 08 17:40:02.240257 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.240232 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-217.ec2.internal" event="NodeReady" Jul 08 17:40:02.240394 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.240382 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 17:40:02.293391 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.293360 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-89lm7"] Jul 08 17:40:02.297250 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.297231 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6pkt6"] Jul 08 17:40:02.297353 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.297342 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-89lm7" Jul 08 17:40:02.300238 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.300215 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 17:40:02.300333 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.300246 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tv27j\"" Jul 08 17:40:02.300333 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.300254 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 17:40:02.300333 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.300248 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 17:40:02.308180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.303948 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.308180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.306513 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 17:40:02.308180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.306971 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-89lm7"] Jul 08 17:40:02.308180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.307376 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 17:40:02.308180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.307698 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 17:40:02.308180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.307852 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p2ql9\"" Jul 08 17:40:02.308180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.307954 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 17:40:02.310654 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.310633 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6pkt6"] Jul 08 17:40:02.344662 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.344637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32cc2e4-7870-4b80-8087-b9d43a8dda10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.344776 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.344704 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32cc2e4-7870-4b80-8087-b9d43a8dda10-data-volume\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.344776 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.344752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32cc2e4-7870-4b80-8087-b9d43a8dda10-crio-socket\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.344848 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.344779 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm2cr\" (UniqueName: \"kubernetes.io/projected/f32cc2e4-7870-4b80-8087-b9d43a8dda10-kube-api-access-mm2cr\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.344848 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.344795 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17943a25-4001-43e7-a778-fa5103f90de2-cert\") pod \"ingress-canary-89lm7\" (UID: \"17943a25-4001-43e7-a778-fa5103f90de2\") " pod="openshift-ingress-canary/ingress-canary-89lm7" Jul 08 17:40:02.344848 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.344817 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32cc2e4-7870-4b80-8087-b9d43a8dda10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.344848 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.344831 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5bxp\" (UniqueName: \"kubernetes.io/projected/17943a25-4001-43e7-a778-fa5103f90de2-kube-api-access-m5bxp\") pod \"ingress-canary-89lm7\" (UID: \"17943a25-4001-43e7-a778-fa5103f90de2\") " pod="openshift-ingress-canary/ingress-canary-89lm7" Jul 08 17:40:02.392803 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.392776 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dgvdn"] Jul 08 17:40:02.401724 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.401704 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.404231 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.404209 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2bzl9\"" Jul 08 17:40:02.404336 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.404210 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 17:40:02.404412 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.404214 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 17:40:02.405262 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.405236 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dgvdn"] Jul 08 17:40:02.445323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445272 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-tmp-dir\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.445323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32cc2e4-7870-4b80-8087-b9d43a8dda10-data-volume\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.445472 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32cc2e4-7870-4b80-8087-b9d43a8dda10-crio-socket\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.445472 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm2cr\" (UniqueName: \"kubernetes.io/projected/f32cc2e4-7870-4b80-8087-b9d43a8dda10-kube-api-access-mm2cr\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.445472 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445426 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32cc2e4-7870-4b80-8087-b9d43a8dda10-crio-socket\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.445472 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17943a25-4001-43e7-a778-fa5103f90de2-cert\") pod \"ingress-canary-89lm7\" (UID: \"17943a25-4001-43e7-a778-fa5103f90de2\") " pod="openshift-ingress-canary/ingress-canary-89lm7" Jul 08 17:40:02.445710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvthv\" (UniqueName: \"kubernetes.io/projected/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-kube-api-access-rvthv\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.445710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32cc2e4-7870-4b80-8087-b9d43a8dda10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.445710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5bxp\" (UniqueName: \"kubernetes.io/projected/17943a25-4001-43e7-a778-fa5103f90de2-kube-api-access-m5bxp\") pod \"ingress-canary-89lm7\" (UID: \"17943a25-4001-43e7-a778-fa5103f90de2\") " pod="openshift-ingress-canary/ingress-canary-89lm7" Jul 08 17:40:02.445710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32cc2e4-7870-4b80-8087-b9d43a8dda10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.445710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445622 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32cc2e4-7870-4b80-8087-b9d43a8dda10-data-volume\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.445710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-metrics-tls\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.445710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.445698 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-config-volume\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.446057 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.446038 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32cc2e4-7870-4b80-8087-b9d43a8dda10-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.449181 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.449164 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32cc2e4-7870-4b80-8087-b9d43a8dda10-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.449300 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.449281 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17943a25-4001-43e7-a778-fa5103f90de2-cert\") pod \"ingress-canary-89lm7\" (UID: \"17943a25-4001-43e7-a778-fa5103f90de2\") " pod="openshift-ingress-canary/ingress-canary-89lm7" Jul 08 17:40:02.455193 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.455172 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67d5d9b9b9-xhv7z"] Jul 08 17:40:02.457224 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.457202 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm2cr\" (UniqueName: \"kubernetes.io/projected/f32cc2e4-7870-4b80-8087-b9d43a8dda10-kube-api-access-mm2cr\") pod \"insights-runtime-extractor-6pkt6\" (UID: \"f32cc2e4-7870-4b80-8087-b9d43a8dda10\") " pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.460207 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.460185 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5bxp\" (UniqueName: \"kubernetes.io/projected/17943a25-4001-43e7-a778-fa5103f90de2-kube-api-access-m5bxp\") pod \"ingress-canary-89lm7\" (UID: \"17943a25-4001-43e7-a778-fa5103f90de2\") " pod="openshift-ingress-canary/ingress-canary-89lm7" Jul 08 17:40:02.469180 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.469159 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.470321 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.470298 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d5d9b9b9-xhv7z"] Jul 08 17:40:02.471647 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.471628 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 17:40:02.471743 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.471677 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 17:40:02.471743 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.471700 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 17:40:02.472161 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.471954 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 17:40:02.472161 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.471960 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 17:40:02.472161 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.471982 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s74jv\"" Jul 08 17:40:02.472161 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.472044 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 17:40:02.472395 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.472235 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 17:40:02.546294 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-config\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.546402 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-metrics-tls\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.546402 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-config-volume\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.546402 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-serving-cert\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.546402 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-tmp-dir\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.546585 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-oauth-config\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.546585 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-oauth-serving-cert\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.546585 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-service-ca\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.546585 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvthv\" (UniqueName: \"kubernetes.io/projected/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-kube-api-access-rvthv\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.546743 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98hdz\" (UniqueName: \"kubernetes.io/projected/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-kube-api-access-98hdz\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.546783 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546769 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-tmp-dir\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.547013 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.546998 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-config-volume\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.548544 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.548529 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-metrics-tls\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.555849 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.555830 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvthv\" (UniqueName: \"kubernetes.io/projected/dffdb6f2-534a-4ffc-be18-f5c7a5246a45-kube-api-access-rvthv\") pod \"dns-default-dgvdn\" (UID: \"dffdb6f2-534a-4ffc-be18-f5c7a5246a45\") " pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.610782 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.610761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-89lm7" Jul 08 17:40:02.617928 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.617911 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6pkt6" Jul 08 17:40:02.647104 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.647080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98hdz\" (UniqueName: \"kubernetes.io/projected/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-kube-api-access-98hdz\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.647262 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.647115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-config\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.647262 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.647164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-serving-cert\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.647262 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.647216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-oauth-config\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.647262 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.647253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-oauth-serving-cert\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.647464 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.647284 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-service-ca\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.648096 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.648073 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-config\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.648252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.648083 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-service-ca\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.648252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.648083 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-oauth-serving-cert\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.649557 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.649535 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-serving-cert\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.650199 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.650175 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-oauth-config\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.657866 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.657750 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98hdz\" (UniqueName: \"kubernetes.io/projected/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-kube-api-access-98hdz\") pod \"console-67d5d9b9b9-xhv7z\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.711703 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.711338 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:02.781022 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.780640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:02.789970 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.789895 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-89lm7"] Jul 08 17:40:02.799892 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.799865 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6pkt6"] Jul 08 17:40:02.801900 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:02.801864 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17943a25_4001_43e7_a778_fa5103f90de2.slice/crio-1b50d9843ca1d9618349532f3a12ddf4e1483e7268d5c849a2c5582b1cd00032 WatchSource:0}: Error finding container 1b50d9843ca1d9618349532f3a12ddf4e1483e7268d5c849a2c5582b1cd00032: Status 404 returned error can't find the container with id 1b50d9843ca1d9618349532f3a12ddf4e1483e7268d5c849a2c5582b1cd00032 Jul 08 17:40:02.803028 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:02.802997 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32cc2e4_7870_4b80_8087_b9d43a8dda10.slice/crio-ef40ac8afdedef7417571635bb2f118ef5b23960e6b9d2d96ff3d052d8214de8 WatchSource:0}: Error finding container ef40ac8afdedef7417571635bb2f118ef5b23960e6b9d2d96ff3d052d8214de8: Status 404 returned error can't find the container with id ef40ac8afdedef7417571635bb2f118ef5b23960e6b9d2d96ff3d052d8214de8 Jul 08 17:40:02.870248 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.870211 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dgvdn"] Jul 08 17:40:02.874043 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:02.874014 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddffdb6f2_534a_4ffc_be18_f5c7a5246a45.slice/crio-92bda4fc16b33ee8d792090ccf56b70a20a0d5cbb8b76bc8f18ca7ef566f2187 WatchSource:0}: Error finding container 92bda4fc16b33ee8d792090ccf56b70a20a0d5cbb8b76bc8f18ca7ef566f2187: Status 404 returned error can't find the container with id 92bda4fc16b33ee8d792090ccf56b70a20a0d5cbb8b76bc8f18ca7ef566f2187 Jul 08 17:40:02.925942 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:02.925912 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d5d9b9b9-xhv7z"] Jul 08 17:40:02.929363 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:02.929341 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92bf3d37_03bc_4c12_ac59_ba433c62f4fc.slice/crio-920934e597bf2680571a415a6b363014343f681313f678e05e9e8bc1533a3d85 WatchSource:0}: Error finding container 920934e597bf2680571a415a6b363014343f681313f678e05e9e8bc1533a3d85: Status 404 returned error can't find the container with id 920934e597bf2680571a415a6b363014343f681313f678e05e9e8bc1533a3d85 Jul 08 17:40:03.039822 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.039585 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d5d9b9b9-xhv7z" event={"ID":"92bf3d37-03bc-4c12-ac59-ba433c62f4fc","Type":"ContainerStarted","Data":"920934e597bf2680571a415a6b363014343f681313f678e05e9e8bc1533a3d85"} Jul 08 17:40:03.040727 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.040706 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6pkt6" event={"ID":"f32cc2e4-7870-4b80-8087-b9d43a8dda10","Type":"ContainerStarted","Data":"ef40ac8afdedef7417571635bb2f118ef5b23960e6b9d2d96ff3d052d8214de8"} Jul 08 17:40:03.041751 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.041733 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dgvdn" event={"ID":"dffdb6f2-534a-4ffc-be18-f5c7a5246a45","Type":"ContainerStarted","Data":"92bda4fc16b33ee8d792090ccf56b70a20a0d5cbb8b76bc8f18ca7ef566f2187"} Jul 08 17:40:03.042756 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.042735 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-89lm7" event={"ID":"17943a25-4001-43e7-a778-fa5103f90de2","Type":"ContainerStarted","Data":"1b50d9843ca1d9618349532f3a12ddf4e1483e7268d5c849a2c5582b1cd00032"} Jul 08 17:40:03.842868 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.841681 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:40:03.842868 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.842327 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:40:03.844679 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.843357 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:40:03.854935 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.850958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 17:40:03.854935 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.851225 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 17:40:03.854935 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.851454 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5skf\"" Jul 08 17:40:03.863728 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.863003 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 17:40:03.863728 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.863080 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jwhsf\"" Jul 08 17:40:03.863728 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:03.863329 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 17:40:04.980101 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:04.979591 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl"] Jul 08 17:40:04.998320 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:04.998292 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl"] Jul 08 17:40:04.998488 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:04.998415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" Jul 08 17:40:05.001038 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:05.001014 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lkkwj\"" Jul 08 17:40:05.001988 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:05.001270 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 17:40:05.067790 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:05.067745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c8231274-d8e9-4502-9312-808207d1dcf0-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-kxrbl\" (UID: \"c8231274-d8e9-4502-9312-808207d1dcf0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" Jul 08 17:40:05.168857 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:05.168824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c8231274-d8e9-4502-9312-808207d1dcf0-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-kxrbl\" (UID: \"c8231274-d8e9-4502-9312-808207d1dcf0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" Jul 08 17:40:05.172838 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:05.172809 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c8231274-d8e9-4502-9312-808207d1dcf0-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-kxrbl\" (UID: \"c8231274-d8e9-4502-9312-808207d1dcf0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" Jul 08 17:40:05.311301 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:05.311065 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" Jul 08 17:40:08.850063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.850029 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-686965cd8f-q6bw4"] Jul 08 17:40:08.857414 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.857394 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:08.870078 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.870052 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-686965cd8f-q6bw4"] Jul 08 17:40:08.871579 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.871543 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 17:40:08.900285 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.900261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-oauth-config\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:08.900421 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.900304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-config\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:08.900421 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.900348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-service-ca\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:08.900421 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.900372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-serving-cert\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:08.900421 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.900394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-trusted-ca-bundle\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:08.900640 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.900425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kgzz\" (UniqueName: \"kubernetes.io/projected/2fcc9853-95b2-4db6-88f2-0ab68f59a052-kube-api-access-5kgzz\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:08.900640 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:08.900545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-oauth-serving-cert\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.001936 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.001899 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kgzz\" (UniqueName: \"kubernetes.io/projected/2fcc9853-95b2-4db6-88f2-0ab68f59a052-kube-api-access-5kgzz\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.002118 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.001949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-oauth-serving-cert\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.002118 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.002015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-oauth-config\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.002118 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.002062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-config\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.002118 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.002101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-service-ca\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.002325 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.002138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-serving-cert\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.002325 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.002163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-trusted-ca-bundle\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.002850 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.002795 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-oauth-serving-cert\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.002972 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.002884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-service-ca\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.003033 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.002973 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-trusted-ca-bundle\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.003033 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.003015 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-config\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.005302 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.005273 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-oauth-config\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.005401 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.005299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-serving-cert\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.011758 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.011735 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kgzz\" (UniqueName: \"kubernetes.io/projected/2fcc9853-95b2-4db6-88f2-0ab68f59a052-kube-api-access-5kgzz\") pod \"console-686965cd8f-q6bw4\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.172990 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.172912 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:09.558409 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.558363 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-686965cd8f-q6bw4"] Jul 08 17:40:09.574236 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:09.573918 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl"] Jul 08 17:40:09.583005 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:09.582965 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8231274_d8e9_4502_9312_808207d1dcf0.slice/crio-683276ac8eb0c47e8be3c3c202a0f81a944ff6bba35eb5dfbf043b8904828f76 WatchSource:0}: Error finding container 683276ac8eb0c47e8be3c3c202a0f81a944ff6bba35eb5dfbf043b8904828f76: Status 404 returned error can't find the container with id 683276ac8eb0c47e8be3c3c202a0f81a944ff6bba35eb5dfbf043b8904828f76 Jul 08 17:40:10.064347 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.064310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dgvdn" event={"ID":"dffdb6f2-534a-4ffc-be18-f5c7a5246a45","Type":"ContainerStarted","Data":"f699ba551d6a68b03df3fe6910ca30d0804f76421a894773ca30e747b13e9c38"} Jul 08 17:40:10.064770 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.064368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dgvdn" event={"ID":"dffdb6f2-534a-4ffc-be18-f5c7a5246a45","Type":"ContainerStarted","Data":"949ca7b8354994468e6f1c7a94e7928c03ef48690bdb99f031d36d424f866bb3"} Jul 08 17:40:10.064770 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.064396 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:10.065793 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.065768 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-89lm7" event={"ID":"17943a25-4001-43e7-a778-fa5103f90de2","Type":"ContainerStarted","Data":"9054f42cc1f39f5f794eff99bff488179070b295b640724aa932977ff90eb94c"} Jul 08 17:40:10.067220 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.067198 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d5d9b9b9-xhv7z" event={"ID":"92bf3d37-03bc-4c12-ac59-ba433c62f4fc","Type":"ContainerStarted","Data":"e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab"} Jul 08 17:40:10.068610 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.068586 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6pkt6" event={"ID":"f32cc2e4-7870-4b80-8087-b9d43a8dda10","Type":"ContainerStarted","Data":"5fecd4fbb0b7dec7f790c915866a97bf4802ae800f4af6f4cc5dbb5e6caffd2d"} Jul 08 17:40:10.069702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.069682 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" event={"ID":"c8231274-d8e9-4502-9312-808207d1dcf0","Type":"ContainerStarted","Data":"683276ac8eb0c47e8be3c3c202a0f81a944ff6bba35eb5dfbf043b8904828f76"} Jul 08 17:40:10.071056 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.071036 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-686965cd8f-q6bw4" event={"ID":"2fcc9853-95b2-4db6-88f2-0ab68f59a052","Type":"ContainerStarted","Data":"089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6"} Jul 08 17:40:10.071133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.071071 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-686965cd8f-q6bw4" event={"ID":"2fcc9853-95b2-4db6-88f2-0ab68f59a052","Type":"ContainerStarted","Data":"d7741a8777b0b49e3b7d30e9653257eff4d782131577e7162bd5f6f91c1106a6"} Jul 08 17:40:10.083557 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.083468 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dgvdn" podStartSLOduration=1.560525053 podStartE2EDuration="8.08345149s" totalImagesPullingTime="6.522926437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:10.083131018 +0000 UTC m=+59.146653850" watchObservedRunningTime="2026-07-08 17:40:10.08345149 +0000 UTC m=+59.146974311" Jul 08 17:40:10.099486 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.099433 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-89lm7" podStartSLOduration=1.505310577 podStartE2EDuration="8.099414839s" totalImagesPullingTime="6.594104262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:10.099180637 +0000 UTC m=+59.162703466" watchObservedRunningTime="2026-07-08 17:40:10.099414839 +0000 UTC m=+59.162937647" Jul 08 17:40:10.117674 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.117175 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67d5d9b9b9-xhv7z" podStartSLOduration=1.645734826 podStartE2EDuration="8.117154042s" totalImagesPullingTime="6.471419216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:10.116490777 +0000 UTC m=+59.180013606" watchObservedRunningTime="2026-07-08 17:40:10.117154042 +0000 UTC m=+59.180676871" Jul 08 17:40:10.134298 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:10.134249 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-686965cd8f-q6bw4" podStartSLOduration=2.134232277 podStartE2EDuration="2.134232277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:10.133029556 +0000 UTC m=+59.196552377" watchObservedRunningTime="2026-07-08 17:40:10.134232277 +0000 UTC m=+59.197755107" Jul 08 17:40:11.075763 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:11.075732 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6pkt6" event={"ID":"f32cc2e4-7870-4b80-8087-b9d43a8dda10","Type":"ContainerStarted","Data":"3b63893ad81b5ed7da4a8d02b6f5875b2dbf2dd652c13f155ffcda0e5e0b9f58"} Jul 08 17:40:12.080562 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:12.080528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" event={"ID":"c8231274-d8e9-4502-9312-808207d1dcf0","Type":"ContainerStarted","Data":"b7f199613258b5d110b6a98f20c9e0c34831a6466d80f3538c52905d817b3da5"} Jul 08 17:40:12.081045 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:12.080775 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" Jul 08 17:40:12.085899 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:12.085874 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" Jul 08 17:40:12.102608 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:12.102557 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kxrbl" podStartSLOduration=6.284892474 podStartE2EDuration="8.102543084s" totalImagesPullingTime="1.81765061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:12.101003512 +0000 UTC m=+61.164526335" watchObservedRunningTime="2026-07-08 17:40:12.102543084 +0000 UTC m=+61.166065891" Jul 08 17:40:12.781011 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:12.780971 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:12.781196 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:12.781044 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:12.782669 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:12.782616 2580 patch_prober.go:28] interesting pod/console-67d5d9b9b9-xhv7z container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 08 17:40:12.782817 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:12.782683 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-67d5d9b9b9-xhv7z" podUID="92bf3d37-03bc-4c12-ac59-ba433c62f4fc" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 08 17:40:13.062414 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.062382 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt"] Jul 08 17:40:13.071238 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.071217 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.074929 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.074458 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 17:40:13.074929 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.074541 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 17:40:13.074929 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.074564 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 17:40:13.074929 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.074777 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 17:40:13.074929 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.074836 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cxzvv\"" Jul 08 17:40:13.075271 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.074966 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 17:40:13.076036 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.076014 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt"] Jul 08 17:40:13.141176 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.140998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09e2efdd-f874-480b-9f43-b4187123c3c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.141176 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.141051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09e2efdd-f874-480b-9f43-b4187123c3c3-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.141176 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.141075 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09e2efdd-f874-480b-9f43-b4187123c3c3-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.141761 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.141231 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2k52\" (UniqueName: \"kubernetes.io/projected/09e2efdd-f874-480b-9f43-b4187123c3c3-kube-api-access-n2k52\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.241915 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.241879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2k52\" (UniqueName: \"kubernetes.io/projected/09e2efdd-f874-480b-9f43-b4187123c3c3-kube-api-access-n2k52\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.242099 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.241930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09e2efdd-f874-480b-9f43-b4187123c3c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.242099 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.241963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09e2efdd-f874-480b-9f43-b4187123c3c3-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.242099 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.241980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09e2efdd-f874-480b-9f43-b4187123c3c3-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.242751 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.242730 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09e2efdd-f874-480b-9f43-b4187123c3c3-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.245011 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.244984 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09e2efdd-f874-480b-9f43-b4187123c3c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.245124 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.245063 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09e2efdd-f874-480b-9f43-b4187123c3c3-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.254989 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.254951 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2k52\" (UniqueName: \"kubernetes.io/projected/09e2efdd-f874-480b-9f43-b4187123c3c3-kube-api-access-n2k52\") pod \"prometheus-operator-567f7b8b7c-6znvt\" (UID: \"09e2efdd-f874-480b-9f43-b4187123c3c3\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.362885 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.362860 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxkx4" Jul 08 17:40:13.382199 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.382182 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" Jul 08 17:40:13.506213 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:13.506180 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt"] Jul 08 17:40:13.511069 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:13.511042 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e2efdd_f874_480b_9f43_b4187123c3c3.slice/crio-d48ee3869be8db5dd9c5604138ba26ad3bf5f26ac1947ad4aa2fff8b6f1000a1 WatchSource:0}: Error finding container d48ee3869be8db5dd9c5604138ba26ad3bf5f26ac1947ad4aa2fff8b6f1000a1: Status 404 returned error can't find the container with id d48ee3869be8db5dd9c5604138ba26ad3bf5f26ac1947ad4aa2fff8b6f1000a1 Jul 08 17:40:14.088848 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:14.088811 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6pkt6" event={"ID":"f32cc2e4-7870-4b80-8087-b9d43a8dda10","Type":"ContainerStarted","Data":"3fd841999027f695dc1af5a8714063e560721fedfae6361f23e94207fa617227"} Jul 08 17:40:14.090048 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:14.090026 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" event={"ID":"09e2efdd-f874-480b-9f43-b4187123c3c3","Type":"ContainerStarted","Data":"d48ee3869be8db5dd9c5604138ba26ad3bf5f26ac1947ad4aa2fff8b6f1000a1"} Jul 08 17:40:14.110157 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:14.110115 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6pkt6" podStartSLOduration=1.761533381 podStartE2EDuration="12.11010218s" totalImagesPullingTime="10.348568799s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:40:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:14.109156214 +0000 UTC m=+63.172679042" watchObservedRunningTime="2026-07-08 17:40:14.11010218 +0000 UTC m=+63.173625009" Jul 08 17:40:16.096742 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:16.096704 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" event={"ID":"09e2efdd-f874-480b-9f43-b4187123c3c3","Type":"ContainerStarted","Data":"e288fb2a7bdd92af959ed261dd67766ec7a3e08425b2f5271b996ef5c2c83056"} Jul 08 17:40:16.097188 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:16.096763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" event={"ID":"09e2efdd-f874-480b-9f43-b4187123c3c3","Type":"ContainerStarted","Data":"48a7b20d0dc88c5d9e25619ddf123b09d0f0d3a64c7114a5c8db0cfc5d40b105"} Jul 08 17:40:16.121077 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:16.121033 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6znvt" podStartSLOduration=1.564297907 podStartE2EDuration="3.121017742s" totalImagesPullingTime="1.556719835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:16.120169781 +0000 UTC m=+65.183692607" watchObservedRunningTime="2026-07-08 17:40:16.121017742 +0000 UTC m=+65.184540571" Jul 08 17:40:17.168164 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.168115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:40:17.168675 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.168182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:40:17.171261 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.171243 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 17:40:17.171997 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.171984 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 17:40:17.180938 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.180907 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/820ed21e-c804-434f-9286-3c201a9b1528-metrics-certs\") pod \"network-metrics-daemon-q7s9l\" (UID: \"820ed21e-c804-434f-9286-3c201a9b1528\") " pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:40:17.180938 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.180924 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/775aa56e-6a57-4992-8079-863793659229-original-pull-secret\") pod \"global-pull-secret-syncer-5jp9r\" (UID: \"775aa56e-6a57-4992-8079-863793659229\") " pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:40:17.269356 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.269307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:40:17.272332 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.272315 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 17:40:17.283023 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.283004 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 17:40:17.293295 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.293274 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dl94\" (UniqueName: \"kubernetes.io/projected/ffc7691f-a570-41ef-b06e-a46d6bf50da1-kube-api-access-5dl94\") pod \"network-check-target-ln4fx\" (UID: \"ffc7691f-a570-41ef-b06e-a46d6bf50da1\") " pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:40:17.368935 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.368908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5jp9r" Jul 08 17:40:17.381298 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.381280 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5skf\"" Jul 08 17:40:17.389507 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.389481 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7s9l" Jul 08 17:40:17.389808 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.389788 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jwhsf\"" Jul 08 17:40:17.397894 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.397874 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:40:17.523905 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.523838 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5jp9r"] Jul 08 17:40:17.527610 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:17.527562 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775aa56e_6a57_4992_8079_863793659229.slice/crio-783f2a570298ced78fd6bd55720b382d87fb2b7bc041969b4a4f861e850f6404 WatchSource:0}: Error finding container 783f2a570298ced78fd6bd55720b382d87fb2b7bc041969b4a4f861e850f6404: Status 404 returned error can't find the container with id 783f2a570298ced78fd6bd55720b382d87fb2b7bc041969b4a4f861e850f6404 Jul 08 17:40:17.544535 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.544511 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q7s9l"] Jul 08 17:40:17.547628 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:17.547600 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod820ed21e_c804_434f_9286_3c201a9b1528.slice/crio-0a86e59c7a8f31ee065eca79d8f4a6bdd8e0f5ec954301277814b537506f02ea WatchSource:0}: Error finding container 0a86e59c7a8f31ee065eca79d8f4a6bdd8e0f5ec954301277814b537506f02ea: Status 404 returned error can't find the container with id 0a86e59c7a8f31ee065eca79d8f4a6bdd8e0f5ec954301277814b537506f02ea Jul 08 17:40:17.560205 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:17.560182 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ln4fx"] Jul 08 17:40:17.564837 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:17.562990 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc7691f_a570_41ef_b06e_a46d6bf50da1.slice/crio-d15e220f7cc8378267815370400260adc76cf579640af2a12f789c001896f662 WatchSource:0}: Error finding container d15e220f7cc8378267815370400260adc76cf579640af2a12f789c001896f662: Status 404 returned error can't find the container with id d15e220f7cc8378267815370400260adc76cf579640af2a12f789c001896f662 Jul 08 17:40:18.104942 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.104890 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ln4fx" event={"ID":"ffc7691f-a570-41ef-b06e-a46d6bf50da1","Type":"ContainerStarted","Data":"d15e220f7cc8378267815370400260adc76cf579640af2a12f789c001896f662"} Jul 08 17:40:18.106423 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.106395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7s9l" event={"ID":"820ed21e-c804-434f-9286-3c201a9b1528","Type":"ContainerStarted","Data":"0a86e59c7a8f31ee065eca79d8f4a6bdd8e0f5ec954301277814b537506f02ea"} Jul 08 17:40:18.107822 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.107753 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5jp9r" event={"ID":"775aa56e-6a57-4992-8079-863793659229","Type":"ContainerStarted","Data":"783f2a570298ced78fd6bd55720b382d87fb2b7bc041969b4a4f861e850f6404"} Jul 08 17:40:18.474908 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.474879 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9"] Jul 08 17:40:18.478755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.478734 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.482367 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.481964 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 17:40:18.482367 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.482206 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 17:40:18.482367 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.482305 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fb792\"" Jul 08 17:40:18.490834 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.490808 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9"] Jul 08 17:40:18.502322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.502298 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k"] Jul 08 17:40:18.505366 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.504787 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.509077 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.508891 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 17:40:18.511703 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.510564 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 17:40:18.511703 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.510857 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 17:40:18.511703 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.511056 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-z8lg7\"" Jul 08 17:40:18.524191 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.524166 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k"] Jul 08 17:40:18.546114 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.546090 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4pcsx"] Jul 08 17:40:18.548452 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.548429 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.551368 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.550714 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 17:40:18.551368 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.550980 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 17:40:18.551985 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.551966 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 17:40:18.552085 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.551967 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cccpp\"" Jul 08 17:40:18.581923 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.581893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctnsx\" (UniqueName: \"kubernetes.io/projected/31e1ec4a-1afc-4796-952f-92f875008da5-kube-api-access-ctnsx\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.582034 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.581942 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.582034 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.582003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e1ec4a-1afc-4796-952f-92f875008da5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.582163 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.582034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e1ec4a-1afc-4796-952f-92f875008da5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.582163 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.582067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.582163 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.582101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvwcd\" (UniqueName: \"kubernetes.io/projected/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-api-access-zvwcd\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.582163 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.582138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.582319 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.582173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e1ec4a-1afc-4796-952f-92f875008da5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.582319 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.582202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e16a73e0-7afc-4605-9412-2c1014bbb981-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.582319 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.582225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e16a73e0-7afc-4605-9412-2c1014bbb981-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.683222 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.683389 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683242 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e1ec4a-1afc-4796-952f-92f875008da5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.683389 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qtlb\" (UniqueName: \"kubernetes.io/projected/174183a0-be00-4cdf-a793-74b5af0b947c-kube-api-access-7qtlb\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.683389 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e16a73e0-7afc-4605-9412-2c1014bbb981-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.683389 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e16a73e0-7afc-4605-9412-2c1014bbb981-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.683389 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683356 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctnsx\" (UniqueName: \"kubernetes.io/projected/31e1ec4a-1afc-4796-952f-92f875008da5-kube-api-access-ctnsx\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.683389 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683383 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-accelerators-collector-config\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-tls\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683470 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/174183a0-be00-4cdf-a793-74b5af0b947c-metrics-client-ca\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-sys\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-root\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e1ec4a-1afc-4796-952f-92f875008da5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e1ec4a-1afc-4796-952f-92f875008da5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-textfile\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.683736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-wtmp\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.684216 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.684216 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.683781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvwcd\" (UniqueName: \"kubernetes.io/projected/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-api-access-zvwcd\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.685940 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.684757 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31e1ec4a-1afc-4796-952f-92f875008da5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.685940 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.685320 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e16a73e0-7afc-4605-9412-2c1014bbb981-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.685940 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.685863 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.685940 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.685905 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e16a73e0-7afc-4605-9412-2c1014bbb981-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.688501 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.687941 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.689769 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.689350 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31e1ec4a-1afc-4796-952f-92f875008da5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.690421 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.690329 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.691088 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.691008 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/31e1ec4a-1afc-4796-952f-92f875008da5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.698069 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.697983 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvwcd\" (UniqueName: \"kubernetes.io/projected/e16a73e0-7afc-4605-9412-2c1014bbb981-kube-api-access-zvwcd\") pod \"kube-state-metrics-7c9dbb8b7d-67j7k\" (UID: \"e16a73e0-7afc-4605-9412-2c1014bbb981\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.698694 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.698668 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctnsx\" (UniqueName: \"kubernetes.io/projected/31e1ec4a-1afc-4796-952f-92f875008da5-kube-api-access-ctnsx\") pod \"openshift-state-metrics-54cf96dfcf-qkfx9\" (UID: \"31e1ec4a-1afc-4796-952f-92f875008da5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.784890 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.784859 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/174183a0-be00-4cdf-a793-74b5af0b947c-metrics-client-ca\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.784901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-sys\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.784923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-root\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785006 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-sys\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-textfile\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785268 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-wtmp\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785268 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785268 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785097 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-root\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785268 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qtlb\" (UniqueName: \"kubernetes.io/projected/174183a0-be00-4cdf-a793-74b5af0b947c-kube-api-access-7qtlb\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785268 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785252 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-wtmp\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785512 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-accelerators-collector-config\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785512 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785318 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-tls\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785512 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:18.785441 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 17:40:18.785512 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:18.785512 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-tls podName:174183a0-be00-4cdf-a793-74b5af0b947c nodeName:}" failed. No retries permitted until 2026-07-08 17:40:19.285493367 +0000 UTC m=+68.349016189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-tls") pod "node-exporter-4pcsx" (UID: "174183a0-be00-4cdf-a793-74b5af0b947c") : secret "node-exporter-tls" not found Jul 08 17:40:18.785743 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785520 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/174183a0-be00-4cdf-a793-74b5af0b947c-metrics-client-ca\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785826 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785806 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-textfile\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.785919 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.785896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-accelerators-collector-config\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.787820 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.787802 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.791936 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.791918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" Jul 08 17:40:18.794467 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.794443 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qtlb\" (UniqueName: \"kubernetes.io/projected/174183a0-be00-4cdf-a793-74b5af0b947c-kube-api-access-7qtlb\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:18.820789 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.820760 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" Jul 08 17:40:18.944602 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.944488 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9"] Jul 08 17:40:18.947854 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:18.947802 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e1ec4a_1afc_4796_952f_92f875008da5.slice/crio-e4a6cf35b1948c19d07bd61dcd697d94a0f456f94594bde5848a04f13f0186ae WatchSource:0}: Error finding container e4a6cf35b1948c19d07bd61dcd697d94a0f456f94594bde5848a04f13f0186ae: Status 404 returned error can't find the container with id e4a6cf35b1948c19d07bd61dcd697d94a0f456f94594bde5848a04f13f0186ae Jul 08 17:40:18.967763 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:18.967281 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k"] Jul 08 17:40:18.969816 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:18.969782 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16a73e0_7afc_4605_9412_2c1014bbb981.slice/crio-4334f6c0a6f466d74647398242a516aad27d20393874ea3427ffc3a5f05d02ff WatchSource:0}: Error finding container 4334f6c0a6f466d74647398242a516aad27d20393874ea3427ffc3a5f05d02ff: Status 404 returned error can't find the container with id 4334f6c0a6f466d74647398242a516aad27d20393874ea3427ffc3a5f05d02ff Jul 08 17:40:19.115493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.115430 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7s9l" event={"ID":"820ed21e-c804-434f-9286-3c201a9b1528","Type":"ContainerStarted","Data":"5411e7b5cedc8b966b2dabaf87073a3a832f5a123a345df3246a98a0d988ef08"} Jul 08 17:40:19.115738 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.115524 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7s9l" event={"ID":"820ed21e-c804-434f-9286-3c201a9b1528","Type":"ContainerStarted","Data":"1ac7aabe5e88c10b69ecc2c1878cf49a998409187373caf7f1df99cc8e34bce3"} Jul 08 17:40:19.117348 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.117314 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" event={"ID":"e16a73e0-7afc-4605-9412-2c1014bbb981","Type":"ContainerStarted","Data":"4334f6c0a6f466d74647398242a516aad27d20393874ea3427ffc3a5f05d02ff"} Jul 08 17:40:19.120277 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.120236 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" event={"ID":"31e1ec4a-1afc-4796-952f-92f875008da5","Type":"ContainerStarted","Data":"e74b1710d1c855bf1ec09a48522291bad27b036ae20763fab8a9437a6da86c30"} Jul 08 17:40:19.120381 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.120316 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" event={"ID":"31e1ec4a-1afc-4796-952f-92f875008da5","Type":"ContainerStarted","Data":"af3d0fc2cfe4ed4a993ea7c2888aca0c90ae53f40c08f22616e994cfc1f1bf12"} Jul 08 17:40:19.120381 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.120346 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" event={"ID":"31e1ec4a-1afc-4796-952f-92f875008da5","Type":"ContainerStarted","Data":"e4a6cf35b1948c19d07bd61dcd697d94a0f456f94594bde5848a04f13f0186ae"} Jul 08 17:40:19.134309 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.134259 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-q7s9l" podStartSLOduration=66.237455926 podStartE2EDuration="1m7.134246579s" totalImagesPullingTime="896.790653ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:19.132883695 +0000 UTC m=+68.196406525" watchObservedRunningTime="2026-07-08 17:40:19.134246579 +0000 UTC m=+68.197769408" Jul 08 17:40:19.173729 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.173618 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:19.173729 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.173670 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:19.175374 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.175342 2580 patch_prober.go:28] interesting pod/console-686965cd8f-q6bw4 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jul 08 17:40:19.175467 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.175388 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-686965cd8f-q6bw4" podUID="2fcc9853-95b2-4db6-88f2-0ab68f59a052" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" Jul 08 17:40:19.290641 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.289392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-tls\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:19.293284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.293251 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/174183a0-be00-4cdf-a793-74b5af0b947c-node-exporter-tls\") pod \"node-exporter-4pcsx\" (UID: \"174183a0-be00-4cdf-a793-74b5af0b947c\") " pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:19.461182 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.461149 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4pcsx" Jul 08 17:40:19.473062 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:19.473022 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod174183a0_be00_4cdf_a793_74b5af0b947c.slice/crio-0a100a0ef965df740517a4420e2b68dc4e95da7e4134776d5be0285b3bfe864c WatchSource:0}: Error finding container 0a100a0ef965df740517a4420e2b68dc4e95da7e4134776d5be0285b3bfe864c: Status 404 returned error can't find the container with id 0a100a0ef965df740517a4420e2b68dc4e95da7e4134776d5be0285b3bfe864c Jul 08 17:40:19.551755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.551171 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:40:19.556447 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.556424 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.560053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.559439 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 17:40:19.560053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.559722 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 17:40:19.560053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.559756 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 17:40:19.560053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.559722 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 17:40:19.560053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.559909 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 17:40:19.560384 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.560099 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 17:40:19.560384 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.560276 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 17:40:19.560384 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.560370 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 17:40:19.560533 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.560469 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-htkcs\"" Jul 08 17:40:19.560616 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.560608 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 17:40:19.579275 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.579249 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:40:19.693616 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693554 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b558aba6-91c4-442a-9f38-3e7955cd2cd7-config-out\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.693616 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693618 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b558aba6-91c4-442a-9f38-3e7955cd2cd7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.693826 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693680 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.693826 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693771 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.693826 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.693951 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-web-config\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.693951 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j7tq\" (UniqueName: \"kubernetes.io/projected/b558aba6-91c4-442a-9f38-3e7955cd2cd7-kube-api-access-6j7tq\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.693951 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b558aba6-91c4-442a-9f38-3e7955cd2cd7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.693951 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693940 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.694091 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.693979 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.694091 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.694053 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-config-volume\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.694172 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.694090 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b558aba6-91c4-442a-9f38-3e7955cd2cd7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.694172 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.694131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b558aba6-91c4-442a-9f38-3e7955cd2cd7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794622 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-web-config\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794697 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j7tq\" (UniqueName: \"kubernetes.io/projected/b558aba6-91c4-442a-9f38-3e7955cd2cd7-kube-api-access-6j7tq\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b558aba6-91c4-442a-9f38-3e7955cd2cd7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-config-volume\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b558aba6-91c4-442a-9f38-3e7955cd2cd7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794916 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b558aba6-91c4-442a-9f38-3e7955cd2cd7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b558aba6-91c4-442a-9f38-3e7955cd2cd7-config-out\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.794975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b558aba6-91c4-442a-9f38-3e7955cd2cd7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.795002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.795864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.795041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.796637 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.796517 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b558aba6-91c4-442a-9f38-3e7955cd2cd7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.797773 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.797744 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b558aba6-91c4-442a-9f38-3e7955cd2cd7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.797900 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:19.797800 2580 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 08 17:40:19.797900 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:19.797865 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-main-tls podName:b558aba6-91c4-442a-9f38-3e7955cd2cd7 nodeName:}" failed. No retries permitted until 2026-07-08 17:40:20.297845868 +0000 UTC m=+69.361368692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "b558aba6-91c4-442a-9f38-3e7955cd2cd7") : secret "alertmanager-main-tls" not found Jul 08 17:40:19.798077 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.798033 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b558aba6-91c4-442a-9f38-3e7955cd2cd7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.803456 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.802788 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-web-config\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.803456 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.803375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.803701 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.803462 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.803926 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.803886 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b558aba6-91c4-442a-9f38-3e7955cd2cd7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.804679 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.804190 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.804679 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.804624 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-config-volume\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.805026 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.804997 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b558aba6-91c4-442a-9f38-3e7955cd2cd7-config-out\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.807063 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.807023 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:19.808077 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:19.808056 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j7tq\" (UniqueName: \"kubernetes.io/projected/b558aba6-91c4-442a-9f38-3e7955cd2cd7-kube-api-access-6j7tq\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:20.079644 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.078811 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dgvdn" Jul 08 17:40:20.128526 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.127977 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4pcsx" event={"ID":"174183a0-be00-4cdf-a793-74b5af0b947c","Type":"ContainerStarted","Data":"0a100a0ef965df740517a4420e2b68dc4e95da7e4134776d5be0285b3bfe864c"} Jul 08 17:40:20.302602 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.301977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:20.313303 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.312218 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b558aba6-91c4-442a-9f38-3e7955cd2cd7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b558aba6-91c4-442a-9f38-3e7955cd2cd7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:20.431447 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.430619 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj"] Jul 08 17:40:20.441658 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.440644 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.444873 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.444789 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 17:40:20.446930 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.445271 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 17:40:20.446930 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.445373 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7xbsd\"" Jul 08 17:40:20.446930 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.445614 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 17:40:20.446930 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.445677 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 17:40:20.446930 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.445872 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 17:40:20.446930 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.446277 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-oifrg9nqp52v\"" Jul 08 17:40:20.455795 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.455774 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj"] Jul 08 17:40:20.474489 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.474454 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:40:20.504431 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.504400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.504579 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.504450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.504579 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.504512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.504702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.504598 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.504702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.504647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-tls\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.504815 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.504701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-metrics-client-ca\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.504815 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.504741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-grpc-tls\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.504815 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.504766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5pr7\" (UniqueName: \"kubernetes.io/projected/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-kube-api-access-v5pr7\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.605769 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.605733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.606234 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.605777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.606234 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.605810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.606234 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.605851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.606234 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.605890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-tls\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.606234 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.605942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-metrics-client-ca\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.606234 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.605981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-grpc-tls\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.606234 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.606012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5pr7\" (UniqueName: \"kubernetes.io/projected/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-kube-api-access-v5pr7\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.607083 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.607024 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-metrics-client-ca\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.609108 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.609056 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-tls\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.609108 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.609093 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.609618 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.609591 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.610428 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.610403 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.610522 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.610475 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.610522 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.610510 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-secret-grpc-tls\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.615905 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.615861 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5pr7\" (UniqueName: \"kubernetes.io/projected/e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f-kube-api-access-v5pr7\") pod \"thanos-querier-6c6c5576cf-5xhcj\" (UID: \"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f\") " pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:20.759068 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:20.758989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:22.757404 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.757369 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b59469cdf-v6z4k"] Jul 08 17:40:22.762315 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.762290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.766086 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.766056 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 17:40:22.766214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.766097 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 17:40:22.766214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.766207 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fbc98\"" Jul 08 17:40:22.766338 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.766275 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 17:40:22.766338 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.766309 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 17:40:22.766450 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.766424 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-359307kj632n1\"" Jul 08 17:40:22.772212 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.772158 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b59469cdf-v6z4k"] Jul 08 17:40:22.781582 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.781535 2580 patch_prober.go:28] interesting pod/console-67d5d9b9b9-xhv7z container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 08 17:40:22.781709 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.781646 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-67d5d9b9b9-xhv7z" podUID="92bf3d37-03bc-4c12-ac59-ba433c62f4fc" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 08 17:40:22.825895 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.825863 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-secret-metrics-server-client-certs\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.826036 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.825901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-secret-metrics-server-tls\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.826036 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.825962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.826036 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.825995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-audit-log\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.826036 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.826021 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-metrics-server-audit-profiles\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.826223 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.826055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psvqh\" (UniqueName: \"kubernetes.io/projected/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-kube-api-access-psvqh\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.826223 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.826095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-client-ca-bundle\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927040 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927040 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-audit-log\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927277 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-metrics-server-audit-profiles\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927277 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927188 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psvqh\" (UniqueName: \"kubernetes.io/projected/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-kube-api-access-psvqh\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927277 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-client-ca-bundle\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927426 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927332 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-secret-metrics-server-client-certs\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927426 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927371 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-secret-metrics-server-tls\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927916 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927543 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-audit-log\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.927916 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.927852 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.928241 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.928175 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-metrics-server-audit-profiles\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.930173 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.930148 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-secret-metrics-server-tls\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.930280 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.930200 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-secret-metrics-server-client-certs\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.930517 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.930494 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-client-ca-bundle\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:22.935997 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:22.935977 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psvqh\" (UniqueName: \"kubernetes.io/projected/bf2401c2-5f64-4cd1-8be4-3b1d7d694684-kube-api-access-psvqh\") pod \"metrics-server-b59469cdf-v6z4k\" (UID: \"bf2401c2-5f64-4cd1-8be4-3b1d7d694684\") " pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:23.074203 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.074167 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:23.214087 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.214054 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw"] Jul 08 17:40:23.233105 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.233081 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw"] Jul 08 17:40:23.233266 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.233166 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:23.235654 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.235627 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 17:40:23.235785 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.235769 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4c2gd\"" Jul 08 17:40:23.330741 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.330667 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-82zbw\" (UID: \"a23c7252-c9bb-4b4c-8ac3-2c57da03b804\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:23.330741 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.330720 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xzj8\" (UniqueName: \"kubernetes.io/projected/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-kube-api-access-9xzj8\") pod \"monitoring-plugin-d5dcff7cf-82zbw\" (UID: \"a23c7252-c9bb-4b4c-8ac3-2c57da03b804\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:23.432039 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.432002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-82zbw\" (UID: \"a23c7252-c9bb-4b4c-8ac3-2c57da03b804\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:23.432214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.432086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xzj8\" (UniqueName: \"kubernetes.io/projected/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-kube-api-access-9xzj8\") pod \"monitoring-plugin-d5dcff7cf-82zbw\" (UID: \"a23c7252-c9bb-4b4c-8ac3-2c57da03b804\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:23.432214 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:23.432173 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 17:40:23.432317 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:23.432259 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-monitoring-plugin-cert podName:a23c7252-c9bb-4b4c-8ac3-2c57da03b804 nodeName:}" failed. No retries permitted until 2026-07-08 17:40:23.932237591 +0000 UTC m=+72.995760416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-82zbw" (UID: "a23c7252-c9bb-4b4c-8ac3-2c57da03b804") : secret "monitoring-plugin-cert" not found Jul 08 17:40:23.441457 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.441434 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xzj8\" (UniqueName: \"kubernetes.io/projected/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-kube-api-access-9xzj8\") pod \"monitoring-plugin-d5dcff7cf-82zbw\" (UID: \"a23c7252-c9bb-4b4c-8ac3-2c57da03b804\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:23.697486 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.697412 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-575d485f8d-g8h49"] Jul 08 17:40:23.712870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.712846 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.716088 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.716054 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 17:40:23.716221 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.716111 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 17:40:23.716221 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.716183 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 17:40:23.716221 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.716184 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 17:40:23.716425 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.716255 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 17:40:23.721744 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.721716 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-np4h7\"" Jul 08 17:40:23.723468 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.723450 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-575d485f8d-g8h49"] Jul 08 17:40:23.724030 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.724012 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 17:40:23.835931 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.835902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-metrics-client-ca\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.836322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.835943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-serving-certs-ca-bundle\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.836322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.835969 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-federate-client-tls\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.836322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.836027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-secret-telemeter-client\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.836322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.836081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.836322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.836113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8rsf\" (UniqueName: \"kubernetes.io/projected/236b6ca0-00c6-4ec3-803d-affd1b02c959-kube-api-access-f8rsf\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.836322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.836140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-telemeter-trusted-ca-bundle\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.836322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.836242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-telemeter-client-tls\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.937121 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-telemeter-client-tls\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.937247 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-82zbw\" (UID: \"a23c7252-c9bb-4b4c-8ac3-2c57da03b804\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:23.937247 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937165 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-metrics-client-ca\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.937247 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-serving-certs-ca-bundle\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.937247 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937223 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-federate-client-tls\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.937386 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-secret-telemeter-client\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.937386 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.937521 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8rsf\" (UniqueName: \"kubernetes.io/projected/236b6ca0-00c6-4ec3-803d-affd1b02c959-kube-api-access-f8rsf\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.937521 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.937437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-telemeter-trusted-ca-bundle\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.938049 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.938012 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-serving-certs-ca-bundle\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.938696 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.938674 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-metrics-client-ca\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.938940 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.938912 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/236b6ca0-00c6-4ec3-803d-affd1b02c959-telemeter-trusted-ca-bundle\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.939954 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.939931 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.940040 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.939976 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a23c7252-c9bb-4b4c-8ac3-2c57da03b804-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-82zbw\" (UID: \"a23c7252-c9bb-4b4c-8ac3-2c57da03b804\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:23.940133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.940111 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-secret-telemeter-client\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.940177 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.940163 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-telemeter-client-tls\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.940220 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.940206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/236b6ca0-00c6-4ec3-803d-affd1b02c959-federate-client-tls\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:23.946644 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:23.946623 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8rsf\" (UniqueName: \"kubernetes.io/projected/236b6ca0-00c6-4ec3-803d-affd1b02c959-kube-api-access-f8rsf\") pod \"telemeter-client-575d485f8d-g8h49\" (UID: \"236b6ca0-00c6-4ec3-803d-affd1b02c959\") " pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:24.024148 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.024088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" Jul 08 17:40:24.143223 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.143204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:24.415692 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.415600 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-575d485f8d-g8h49"] Jul 08 17:40:24.425779 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:24.425550 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236b6ca0_00c6_4ec3_803d_affd1b02c959.slice/crio-8f7ffcde43f54ba4214720cef8c4af9c1510b635d7fa463cce76ae91db153c83 WatchSource:0}: Error finding container 8f7ffcde43f54ba4214720cef8c4af9c1510b635d7fa463cce76ae91db153c83: Status 404 returned error can't find the container with id 8f7ffcde43f54ba4214720cef8c4af9c1510b635d7fa463cce76ae91db153c83 Jul 08 17:40:24.491239 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.491113 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw"] Jul 08 17:40:24.497259 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.496656 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d5d9b9b9-xhv7z"] Jul 08 17:40:24.515342 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:24.515313 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23c7252_c9bb_4b4c_8ac3_2c57da03b804.slice/crio-fe8dcfc97080902e37914a71524f53dfdefe160eff784b35d1b63010ed46311f WatchSource:0}: Error finding container fe8dcfc97080902e37914a71524f53dfdefe160eff784b35d1b63010ed46311f: Status 404 returned error can't find the container with id fe8dcfc97080902e37914a71524f53dfdefe160eff784b35d1b63010ed46311f Jul 08 17:40:24.544315 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.544252 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68895d8676-sl25s"] Jul 08 17:40:24.548438 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.547556 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.548438 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.548348 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68895d8676-sl25s"] Jul 08 17:40:24.645100 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.645071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-config\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.646013 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.645986 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-service-ca\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.646204 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.646183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-oauth-config\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.646346 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.646330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-trusted-ca-bundle\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.646473 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.646456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-serving-cert\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.646473 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.646486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-oauth-serving-cert\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.646645 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.646552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb9ns\" (UniqueName: \"kubernetes.io/projected/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-kube-api-access-kb9ns\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.647633 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.647614 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b59469cdf-v6z4k"] Jul 08 17:40:24.649469 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:24.649421 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf2401c2_5f64_4cd1_8be4_3b1d7d694684.slice/crio-d60ba877a677639af7b39884f6e281d9dcd09bcc08f99af78497b38d9c882be3 WatchSource:0}: Error finding container d60ba877a677639af7b39884f6e281d9dcd09bcc08f99af78497b38d9c882be3: Status 404 returned error can't find the container with id d60ba877a677639af7b39884f6e281d9dcd09bcc08f99af78497b38d9c882be3 Jul 08 17:40:24.666118 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.666085 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj"] Jul 08 17:40:24.669371 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.669350 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:40:24.672542 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:24.672520 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fa2c47_aad4_4f89_a85f_a6be5b4c8f3f.slice/crio-e373d2ab94c07ffbde934e1de281639f19713449a0be03f5e04453bb256c1901 WatchSource:0}: Error finding container e373d2ab94c07ffbde934e1de281639f19713449a0be03f5e04453bb256c1901: Status 404 returned error can't find the container with id e373d2ab94c07ffbde934e1de281639f19713449a0be03f5e04453bb256c1901 Jul 08 17:40:24.748094 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.748050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-service-ca\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.748282 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.748103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-oauth-config\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.748282 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.748133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-trusted-ca-bundle\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.748282 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.748163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-serving-cert\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.748282 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.748188 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-oauth-serving-cert\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.748282 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.748246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb9ns\" (UniqueName: \"kubernetes.io/projected/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-kube-api-access-kb9ns\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.748562 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.748304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-config\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.749133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.749103 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-config\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.749133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.749121 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-service-ca\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.749260 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.749213 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-oauth-serving-cert\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.749260 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.749232 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-trusted-ca-bundle\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.751077 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.751052 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-serving-cert\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.751468 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.751451 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-oauth-config\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.760976 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.760947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb9ns\" (UniqueName: \"kubernetes.io/projected/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-kube-api-access-kb9ns\") pod \"console-68895d8676-sl25s\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.816331 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.816295 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:40:24.820671 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.820067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.823123 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.823098 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 17:40:24.823256 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.823098 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 17:40:24.823487 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.823331 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 17:40:24.823487 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.823340 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 17:40:24.823487 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.823398 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 17:40:24.823487 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.823470 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 17:40:24.823806 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.823791 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8m00g1uhlsd2a\"" Jul 08 17:40:24.824302 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.824284 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 17:40:24.824601 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.824555 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7r59t\"" Jul 08 17:40:24.824601 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.824597 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 17:40:24.825480 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.825364 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 17:40:24.825626 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.825606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 17:40:24.825714 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.825702 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 17:40:24.825967 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.825813 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 17:40:24.827983 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.827506 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 17:40:24.831185 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.831164 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 17:40:24.843084 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.843050 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:40:24.912111 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.912078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:24.950930 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.950893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951071 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.950936 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951071 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.950963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951071 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951008 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951071 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-config\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951115 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951137 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951212 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3921707f-3ea9-4e71-b584-5777cd12edc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951498 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951281 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3921707f-3ea9-4e71-b584-5777cd12edc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951498 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951498 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx455\" (UniqueName: \"kubernetes.io/projected/3921707f-3ea9-4e71-b584-5777cd12edc4-kube-api-access-fx455\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951498 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951498 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951415 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951498 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951498 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951467 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951745 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951510 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:24.951745 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:24.951540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052257 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052257 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-config\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052376 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3921707f-3ea9-4e71-b584-5777cd12edc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3921707f-3ea9-4e71-b584-5777cd12edc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052493 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052495 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx455\" (UniqueName: \"kubernetes.io/projected/3921707f-3ea9-4e71-b584-5777cd12edc4-kube-api-access-fx455\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052555 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052638 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.052917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.052767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.053409 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.053138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.053409 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.053345 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.053970 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.053946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.055830 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.055388 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-config\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.055830 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.055774 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.057387 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.057352 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.057853 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.057827 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.058994 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.058746 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-web-config\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.059190 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.059169 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.059435 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.059411 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.059564 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.059540 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.059668 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.059628 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.060838 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.059911 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.060838 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.060444 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3921707f-3ea9-4e71-b584-5777cd12edc4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.061172 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.061066 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3921707f-3ea9-4e71-b584-5777cd12edc4-config-out\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.061338 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.061318 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3921707f-3ea9-4e71-b584-5777cd12edc4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.061417 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.061365 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68895d8676-sl25s"] Jul 08 17:40:25.061776 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.061756 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.062467 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.062443 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.063132 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.063103 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3921707f-3ea9-4e71-b584-5777cd12edc4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.064169 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:25.064144 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f5dde7_5633_413d_a8f6_0ab505c5faa1.slice/crio-fd07b760faeda99237920beda73ad004633fd07a476a2b6c750ce021a022ca63 WatchSource:0}: Error finding container fd07b760faeda99237920beda73ad004633fd07a476a2b6c750ce021a022ca63: Status 404 returned error can't find the container with id fd07b760faeda99237920beda73ad004633fd07a476a2b6c750ce021a022ca63 Jul 08 17:40:25.069320 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.069275 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx455\" (UniqueName: \"kubernetes.io/projected/3921707f-3ea9-4e71-b584-5777cd12edc4-kube-api-access-fx455\") pod \"prometheus-k8s-0\" (UID: \"3921707f-3ea9-4e71-b584-5777cd12edc4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.136514 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.136486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:25.159960 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.159925 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5jp9r" event={"ID":"775aa56e-6a57-4992-8079-863793659229","Type":"ContainerStarted","Data":"2689d0e525a35972e7e31a5c3dc07847d07f276968adddc90c04ace712afd523"} Jul 08 17:40:25.161852 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.161821 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" event={"ID":"236b6ca0-00c6-4ec3-803d-affd1b02c959","Type":"ContainerStarted","Data":"8f7ffcde43f54ba4214720cef8c4af9c1510b635d7fa463cce76ae91db153c83"} Jul 08 17:40:25.163261 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.163232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" event={"ID":"bf2401c2-5f64-4cd1-8be4-3b1d7d694684","Type":"ContainerStarted","Data":"d60ba877a677639af7b39884f6e281d9dcd09bcc08f99af78497b38d9c882be3"} Jul 08 17:40:25.166120 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.166097 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" event={"ID":"e16a73e0-7afc-4605-9412-2c1014bbb981","Type":"ContainerStarted","Data":"fb772d5b9f47bad7e7aa62156656681ad0ba87b980a58360bd35b8275ce0be5d"} Jul 08 17:40:25.166246 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.166234 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" event={"ID":"e16a73e0-7afc-4605-9412-2c1014bbb981","Type":"ContainerStarted","Data":"a2dab87a31450966db832ceb97dc215b0c73f4a1c0bd13515f7ead4ae80189e5"} Jul 08 17:40:25.166332 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.166322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" event={"ID":"e16a73e0-7afc-4605-9412-2c1014bbb981","Type":"ContainerStarted","Data":"deae7e0ce3c2d2f03fd344f5c4ad405850b7519eb5dc92fa8b136fa78b6cbdf0"} Jul 08 17:40:25.168346 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.168328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" event={"ID":"31e1ec4a-1afc-4796-952f-92f875008da5","Type":"ContainerStarted","Data":"d477609df249403bb18e2f0289594216a6aa8b5bc114baa400039423a9e1452b"} Jul 08 17:40:25.169704 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.169687 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" event={"ID":"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f","Type":"ContainerStarted","Data":"e373d2ab94c07ffbde934e1de281639f19713449a0be03f5e04453bb256c1901"} Jul 08 17:40:25.171000 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.170984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" event={"ID":"a23c7252-c9bb-4b4c-8ac3-2c57da03b804","Type":"ContainerStarted","Data":"fe8dcfc97080902e37914a71524f53dfdefe160eff784b35d1b63010ed46311f"} Jul 08 17:40:25.173511 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.173493 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ln4fx" event={"ID":"ffc7691f-a570-41ef-b06e-a46d6bf50da1","Type":"ContainerStarted","Data":"24834d4746894d2eb5693a02769b6180e74ba902a862139785114a5f3168f006"} Jul 08 17:40:25.174025 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.173797 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:40:25.176311 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.175817 2580 generic.go:356] "Generic (PLEG): container finished" podID="174183a0-be00-4cdf-a793-74b5af0b947c" containerID="af70cbf0b1d253e43ae4d34de6115ac228682d47b16c1b55002d9d925bd3e8a1" exitCode=0 Jul 08 17:40:25.176311 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.176259 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4pcsx" event={"ID":"174183a0-be00-4cdf-a793-74b5af0b947c","Type":"ContainerDied","Data":"af70cbf0b1d253e43ae4d34de6115ac228682d47b16c1b55002d9d925bd3e8a1"} Jul 08 17:40:25.177978 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.177936 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5jp9r" podStartSLOduration=66.503290881 podStartE2EDuration="1m13.177919398s" totalImagesPullingTime="6.674628517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:25.175600382 +0000 UTC m=+74.239123210" watchObservedRunningTime="2026-07-08 17:40:25.177919398 +0000 UTC m=+74.241442228" Jul 08 17:40:25.178775 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.178479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68895d8676-sl25s" event={"ID":"e4f5dde7-5633-413d-a8f6-0ab505c5faa1","Type":"ContainerStarted","Data":"ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be"} Jul 08 17:40:25.178972 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.178952 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68895d8676-sl25s" event={"ID":"e4f5dde7-5633-413d-a8f6-0ab505c5faa1","Type":"ContainerStarted","Data":"fd07b760faeda99237920beda73ad004633fd07a476a2b6c750ce021a022ca63"} Jul 08 17:40:25.180129 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.180108 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b558aba6-91c4-442a-9f38-3e7955cd2cd7","Type":"ContainerStarted","Data":"dcc8df57719371e4e5c5b3da7a3ebf51780a3d46f4153c73a3cf2b303cf771ef"} Jul 08 17:40:25.194065 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.194016 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-67j7k" podStartSLOduration=1.964803114 podStartE2EDuration="7.194000113s" totalImagesPullingTime="5.229196999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:25.193470254 +0000 UTC m=+74.256993085" watchObservedRunningTime="2026-07-08 17:40:25.194000113 +0000 UTC m=+74.257522943" Jul 08 17:40:25.215630 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.215489 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ln4fx" podStartSLOduration=66.583842273 podStartE2EDuration="1m13.215467638s" totalImagesPullingTime="6.631625365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:25.214044953 +0000 UTC m=+74.277567782" watchObservedRunningTime="2026-07-08 17:40:25.215467638 +0000 UTC m=+74.278990468" Jul 08 17:40:25.235105 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.235044 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qkfx9" podStartSLOduration=2.107920157 podStartE2EDuration="7.235022523s" totalImagesPullingTime="5.127102366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:25.231993078 +0000 UTC m=+74.295515908" watchObservedRunningTime="2026-07-08 17:40:25.235022523 +0000 UTC m=+74.298545354" Jul 08 17:40:25.271067 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.270727 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68895d8676-sl25s" podStartSLOduration=1.270705086 podStartE2EDuration="1.270705086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:25.26975366 +0000 UTC m=+74.333276488" watchObservedRunningTime="2026-07-08 17:40:25.270705086 +0000 UTC m=+74.334227915" Jul 08 17:40:25.310509 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:25.310008 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:40:25.318002 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:25.317666 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3921707f_3ea9_4e71_b584_5777cd12edc4.slice/crio-e6f4ccc06a85bd8eaca655d17d89853edb392c755ee8b314d0d2b213b6491bb8 WatchSource:0}: Error finding container e6f4ccc06a85bd8eaca655d17d89853edb392c755ee8b314d0d2b213b6491bb8: Status 404 returned error can't find the container with id e6f4ccc06a85bd8eaca655d17d89853edb392c755ee8b314d0d2b213b6491bb8 Jul 08 17:40:26.197436 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:26.197163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4pcsx" event={"ID":"174183a0-be00-4cdf-a793-74b5af0b947c","Type":"ContainerStarted","Data":"a0bffcb1b4a87d07c14f71a06ca0aa60719cc6fbd630a83d077ddb042257c198"} Jul 08 17:40:26.197436 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:26.197229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4pcsx" event={"ID":"174183a0-be00-4cdf-a793-74b5af0b947c","Type":"ContainerStarted","Data":"6069ef7ffd715d999c8b26321977c61546cf7a29818034c713deeed72666993f"} Jul 08 17:40:26.203504 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:26.203361 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3921707f-3ea9-4e71-b584-5777cd12edc4","Type":"ContainerStarted","Data":"e6f4ccc06a85bd8eaca655d17d89853edb392c755ee8b314d0d2b213b6491bb8"} Jul 08 17:40:26.217972 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:26.217912 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4pcsx" podStartSLOduration=3.307862757 podStartE2EDuration="8.217874737s" totalImagesPullingTime="4.721629842s" totalInitContainerRuntime="188.382138ms" isStatefulPod=true podCreationTimestamp="2026-07-08 17:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:26.217147927 +0000 UTC m=+75.280670759" watchObservedRunningTime="2026-07-08 17:40:26.217874737 +0000 UTC m=+75.281397567" Jul 08 17:40:29.178199 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.178163 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:29.182283 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.182255 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:40:29.225078 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.225006 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" event={"ID":"236b6ca0-00c6-4ec3-803d-affd1b02c959","Type":"ContainerStarted","Data":"202aa366819f02e390a8f1a75e7cd15412b9319196bea272f7443ce4a5e63d30"} Jul 08 17:40:29.225404 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.225320 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" event={"ID":"236b6ca0-00c6-4ec3-803d-affd1b02c959","Type":"ContainerStarted","Data":"c14721adb943ead0f19965e1259550d111933211348283bda0428c75315fbb98"} Jul 08 17:40:29.225404 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.225372 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" event={"ID":"236b6ca0-00c6-4ec3-803d-affd1b02c959","Type":"ContainerStarted","Data":"fb8b2012efe52d52cea506e3b1be00069367348cdeea128a6b3559ddb9eaee66"} Jul 08 17:40:29.226926 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.226888 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" event={"ID":"bf2401c2-5f64-4cd1-8be4-3b1d7d694684","Type":"ContainerStarted","Data":"b8627b43bac5ffd42667f4831a9eb587e82d0a2697023c9bd8d70656d721f2b1"} Jul 08 17:40:29.229340 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.229312 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" event={"ID":"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f","Type":"ContainerStarted","Data":"ab561e6f6cc6a54ff0dd8be219f201fe2d521b76d1367a8ebabe1dddac2a71e5"} Jul 08 17:40:29.229434 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.229379 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" event={"ID":"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f","Type":"ContainerStarted","Data":"116845c3e4749b1a00518f7f269c404913913a9f2e8695305616a7e701b58bf1"} Jul 08 17:40:29.229434 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.229417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" event={"ID":"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f","Type":"ContainerStarted","Data":"f46d8ef32b9f10b31e74671b1fe4c97b68c5b50212e408697737369f7cc23e5f"} Jul 08 17:40:29.231224 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.231198 2580 generic.go:356] "Generic (PLEG): container finished" podID="3921707f-3ea9-4e71-b584-5777cd12edc4" containerID="e1700ca6bc7c4329a18cc9d921f945e25f5d874c52b846030ad8ad133d442722" exitCode=0 Jul 08 17:40:29.231330 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.231289 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3921707f-3ea9-4e71-b584-5777cd12edc4","Type":"ContainerDied","Data":"e1700ca6bc7c4329a18cc9d921f945e25f5d874c52b846030ad8ad133d442722"} Jul 08 17:40:29.233165 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.233036 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" event={"ID":"a23c7252-c9bb-4b4c-8ac3-2c57da03b804","Type":"ContainerStarted","Data":"724632c5edb8bd575407934eb4bea06ba1761b2ec757eece3f9716277e5c0e2b"} Jul 08 17:40:29.233233 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.233219 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:29.234858 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.234832 2580 generic.go:356] "Generic (PLEG): container finished" podID="b558aba6-91c4-442a-9f38-3e7955cd2cd7" containerID="a479eb4add89bade24ca095381a78085cdfa189d16578c0e43bf6e12983d25bc" exitCode=0 Jul 08 17:40:29.234993 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.234974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b558aba6-91c4-442a-9f38-3e7955cd2cd7","Type":"ContainerDied","Data":"a479eb4add89bade24ca095381a78085cdfa189d16578c0e43bf6e12983d25bc"} Jul 08 17:40:29.240247 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.240226 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" Jul 08 17:40:29.255692 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.255646 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-575d485f8d-g8h49" podStartSLOduration=2.393209667 podStartE2EDuration="6.255631097s" totalImagesPullingTime="3.86242143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:29.253920513 +0000 UTC m=+78.317443369" watchObservedRunningTime="2026-07-08 17:40:29.255631097 +0000 UTC m=+78.319153905" Jul 08 17:40:29.278889 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.278818 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" podStartSLOduration=3.641130494 podStartE2EDuration="7.278801422s" totalImagesPullingTime="3.637670928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:29.27706579 +0000 UTC m=+78.340588620" watchObservedRunningTime="2026-07-08 17:40:29.278801422 +0000 UTC m=+78.342324228" Jul 08 17:40:29.360297 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:29.360257 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-82zbw" podStartSLOduration=2.595994853 podStartE2EDuration="6.360243136s" totalImagesPullingTime="3.764248283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:29.358806008 +0000 UTC m=+78.422328838" watchObservedRunningTime="2026-07-08 17:40:29.360243136 +0000 UTC m=+78.423765965" Jul 08 17:40:30.248318 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.247068 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" event={"ID":"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f","Type":"ContainerStarted","Data":"b14e9ba98225dfcfe53751e1a0dd072a3eda88c297061a29fd25cdc29b643e0f"} Jul 08 17:40:30.248318 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.247146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" event={"ID":"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f","Type":"ContainerStarted","Data":"d4787db43d10184493dada6531addb8555edc94b3d863f4ac9a4c47519e0832b"} Jul 08 17:40:30.248318 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.247184 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" event={"ID":"e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f","Type":"ContainerStarted","Data":"2711d248c5a0906bc8f6f0567daaf15c45e42fa277cc778e5df57be26661e7ca"} Jul 08 17:40:30.248318 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.248250 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:30.272052 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.271732 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" podStartSLOduration=5.980829173 podStartE2EDuration="10.271711821s" totalImagesPullingTime="4.290882648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:30.271648281 +0000 UTC m=+79.335171113" watchObservedRunningTime="2026-07-08 17:40:30.271711821 +0000 UTC m=+79.335234653" Jul 08 17:40:30.386731 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.386689 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68895d8676-sl25s"] Jul 08 17:40:30.426817 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.426786 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-597bcc946f-67dc5"] Jul 08 17:40:30.431962 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.431935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.445658 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.445630 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-597bcc946f-67dc5"] Jul 08 17:40:30.514774 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.514682 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-oauth-config\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.514774 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.514757 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-serving-cert\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.514991 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.514790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2k4c\" (UniqueName: \"kubernetes.io/projected/2e665281-fd4d-4226-bd97-409d08558ce7-kube-api-access-g2k4c\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.514991 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.514837 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-console-config\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.514991 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.514864 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-oauth-serving-cert\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.514991 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.514900 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-service-ca\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.514991 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.514930 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-trusted-ca-bundle\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.615890 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.615852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-oauth-config\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.616061 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.615939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-serving-cert\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.616061 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.615965 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2k4c\" (UniqueName: \"kubernetes.io/projected/2e665281-fd4d-4226-bd97-409d08558ce7-kube-api-access-g2k4c\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.616061 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.616001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-console-config\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.616061 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.616027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-oauth-serving-cert\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.616270 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.616066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-service-ca\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.616270 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.616101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-trusted-ca-bundle\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.616881 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.616850 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-oauth-serving-cert\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.617002 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.616889 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-console-config\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.617072 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.617050 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-trusted-ca-bundle\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.617133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.617073 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-service-ca\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.618662 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.618643 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-serving-cert\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.619270 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.619250 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-oauth-config\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.626605 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.626539 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2k4c\" (UniqueName: \"kubernetes.io/projected/2e665281-fd4d-4226-bd97-409d08558ce7-kube-api-access-g2k4c\") pod \"console-597bcc946f-67dc5\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.743080 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.743052 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:30.903074 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:30.903047 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-597bcc946f-67dc5"] Jul 08 17:40:30.907789 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:40:30.907761 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e665281_fd4d_4226_bd97_409d08558ce7.slice/crio-660d8b1fb871674304286f5d2f6852e0c5267672e29f92d186d92ed87fdf98b1 WatchSource:0}: Error finding container 660d8b1fb871674304286f5d2f6852e0c5267672e29f92d186d92ed87fdf98b1: Status 404 returned error can't find the container with id 660d8b1fb871674304286f5d2f6852e0c5267672e29f92d186d92ed87fdf98b1 Jul 08 17:40:31.252412 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.252380 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597bcc946f-67dc5" event={"ID":"2e665281-fd4d-4226-bd97-409d08558ce7","Type":"ContainerStarted","Data":"65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd"} Jul 08 17:40:31.252865 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.252437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597bcc946f-67dc5" event={"ID":"2e665281-fd4d-4226-bd97-409d08558ce7","Type":"ContainerStarted","Data":"660d8b1fb871674304286f5d2f6852e0c5267672e29f92d186d92ed87fdf98b1"} Jul 08 17:40:31.256891 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.256858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b558aba6-91c4-442a-9f38-3e7955cd2cd7","Type":"ContainerStarted","Data":"67e6896abc0c09af46ce5cd3448b97620df85cd76cc6992fc894a56597887c58"} Jul 08 17:40:31.257031 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.256950 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b558aba6-91c4-442a-9f38-3e7955cd2cd7","Type":"ContainerStarted","Data":"a1c0cb64f6742cedcdf2b177890209539e178b86c831f763581835a94617aa3a"} Jul 08 17:40:31.257031 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.256995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b558aba6-91c4-442a-9f38-3e7955cd2cd7","Type":"ContainerStarted","Data":"5b76f98e02c7fb6fea278700d395d13ae93e633a54a04329b70f467fb7c4a6a3"} Jul 08 17:40:31.257143 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.257033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b558aba6-91c4-442a-9f38-3e7955cd2cd7","Type":"ContainerStarted","Data":"2ee0618cd55e78d5dc5f6d739685bff651c4e227d697ee1ee60a9e8caabe90ed"} Jul 08 17:40:31.257143 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.257072 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b558aba6-91c4-442a-9f38-3e7955cd2cd7","Type":"ContainerStarted","Data":"cfbb2862d67ff9b587aec95cc506c4ff713121cc4676b60f1623d0a47673037d"} Jul 08 17:40:31.257143 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.257113 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b558aba6-91c4-442a-9f38-3e7955cd2cd7","Type":"ContainerStarted","Data":"fbf47b45773340608f935dbd1a4ce5d50e020f1c53796e5af350343be485bd5d"} Jul 08 17:40:31.276649 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.276538 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-597bcc946f-67dc5" podStartSLOduration=1.276523556 podStartE2EDuration="1.276523556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:31.274543461 +0000 UTC m=+80.338066299" watchObservedRunningTime="2026-07-08 17:40:31.276523556 +0000 UTC m=+80.340046428" Jul 08 17:40:31.309203 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:31.309141 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=7.243338692 podStartE2EDuration="12.309123444s" totalImagesPullingTime="5.033884888s" totalInitContainerRuntime="31.899864ms" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:31.308895128 +0000 UTC m=+80.372417973" watchObservedRunningTime="2026-07-08 17:40:31.309123444 +0000 UTC m=+80.372646273" Jul 08 17:40:33.269829 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:33.269794 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3921707f-3ea9-4e71-b584-5777cd12edc4","Type":"ContainerStarted","Data":"3bab174949f0d6f7802903303cbeb0bc470d505985056d2d5a0e20c8ad908924"} Jul 08 17:40:33.270249 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:33.269857 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3921707f-3ea9-4e71-b584-5777cd12edc4","Type":"ContainerStarted","Data":"1e9134356addde7be1a59a977b6197dd65381776f7ea69bd486405859de56a10"} Jul 08 17:40:33.270249 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:33.269884 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3921707f-3ea9-4e71-b584-5777cd12edc4","Type":"ContainerStarted","Data":"2a13d43908c70b3f26a608ecce7637de19abc0706ebb0872fe1c277e2a475e2d"} Jul 08 17:40:33.270249 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:33.269909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3921707f-3ea9-4e71-b584-5777cd12edc4","Type":"ContainerStarted","Data":"b2858581e827fa6d471cd105c30f00ed881187da190ca4be8c85795bec193576"} Jul 08 17:40:33.270249 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:33.269938 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3921707f-3ea9-4e71-b584-5777cd12edc4","Type":"ContainerStarted","Data":"9d910a8061a0641970915aa7f1b77feabb2c05821e8c575a52a4b4133a9e8743"} Jul 08 17:40:33.270249 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:33.269962 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3921707f-3ea9-4e71-b584-5777cd12edc4","Type":"ContainerStarted","Data":"183b18145a194fab3a265bcf49d697d09e0fded1c3d6a169d08b34d455a639a9"} Jul 08 17:40:33.313764 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:33.313715 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.724373355 podStartE2EDuration="9.313697654s" totalImagesPullingTime="6.557125372s" totalInitContainerRuntime="32.198927ms" isStatefulPod=false podCreationTimestamp="2026-07-08 17:40:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:33.310253639 +0000 UTC m=+82.373776468" watchObservedRunningTime="2026-07-08 17:40:33.313697654 +0000 UTC m=+82.377220483" Jul 08 17:40:34.912657 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:34.912608 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:35.137129 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:35.137094 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:40:37.268347 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:37.268318 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c6c5576cf-5xhcj" Jul 08 17:40:40.744398 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:40.744363 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:40.744806 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:40.744442 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:40.749140 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:40.749119 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:41.309081 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:41.309055 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:40:41.381911 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:41.381877 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-686965cd8f-q6bw4"] Jul 08 17:40:43.074293 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:43.074261 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:43.074683 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:43.074330 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:40:49.536213 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.536172 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-67d5d9b9b9-xhv7z" podUID="92bf3d37-03bc-4c12-ac59-ba433c62f4fc" containerName="console" containerID="cri-o://e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab" gracePeriod=15 Jul 08 17:40:49.779011 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.778988 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:49.889863 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.889839 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-oauth-serving-cert\") pod \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " Jul 08 17:40:49.890044 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.889908 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-service-ca\") pod \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " Jul 08 17:40:49.890044 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.889945 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-oauth-config\") pod \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " Jul 08 17:40:49.890044 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.889971 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-config\") pod \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " Jul 08 17:40:49.890044 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.890005 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98hdz\" (UniqueName: \"kubernetes.io/projected/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-kube-api-access-98hdz\") pod \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " Jul 08 17:40:49.890272 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.890056 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-serving-cert\") pod \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\" (UID: \"92bf3d37-03bc-4c12-ac59-ba433c62f4fc\") " Jul 08 17:40:49.890363 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.890339 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-service-ca" (OuterVolumeSpecName: "service-ca") pod "92bf3d37-03bc-4c12-ac59-ba433c62f4fc" (UID: "92bf3d37-03bc-4c12-ac59-ba433c62f4fc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:40:49.890424 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.890338 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "92bf3d37-03bc-4c12-ac59-ba433c62f4fc" (UID: "92bf3d37-03bc-4c12-ac59-ba433c62f4fc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:40:49.890424 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.890393 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-config" (OuterVolumeSpecName: "console-config") pod "92bf3d37-03bc-4c12-ac59-ba433c62f4fc" (UID: "92bf3d37-03bc-4c12-ac59-ba433c62f4fc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:40:49.892351 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.892315 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "92bf3d37-03bc-4c12-ac59-ba433c62f4fc" (UID: "92bf3d37-03bc-4c12-ac59-ba433c62f4fc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:40:49.892351 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.892338 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-kube-api-access-98hdz" (OuterVolumeSpecName: "kube-api-access-98hdz") pod "92bf3d37-03bc-4c12-ac59-ba433c62f4fc" (UID: "92bf3d37-03bc-4c12-ac59-ba433c62f4fc"). InnerVolumeSpecName "kube-api-access-98hdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:40:49.892486 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.892375 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "92bf3d37-03bc-4c12-ac59-ba433c62f4fc" (UID: "92bf3d37-03bc-4c12-ac59-ba433c62f4fc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:40:49.991496 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.991463 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-service-ca\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:49.991496 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.991499 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-oauth-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:49.991739 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.991514 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:49.991739 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.991529 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98hdz\" (UniqueName: \"kubernetes.io/projected/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-kube-api-access-98hdz\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:49.991739 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.991544 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-console-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:49.991739 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:49.991555 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92bf3d37-03bc-4c12-ac59-ba433c62f4fc-oauth-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:50.347920 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.347881 2580 generic.go:356] "Generic (PLEG): container finished" podID="92bf3d37-03bc-4c12-ac59-ba433c62f4fc" containerID="e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab" exitCode=0 Jul 08 17:40:50.347920 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.347922 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d5d9b9b9-xhv7z" Jul 08 17:40:50.348313 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.347967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d5d9b9b9-xhv7z" event={"ID":"92bf3d37-03bc-4c12-ac59-ba433c62f4fc","Type":"ContainerDied","Data":"e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab"} Jul 08 17:40:50.348313 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.348016 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d5d9b9b9-xhv7z" event={"ID":"92bf3d37-03bc-4c12-ac59-ba433c62f4fc","Type":"ContainerDied","Data":"920934e597bf2680571a415a6b363014343f681313f678e05e9e8bc1533a3d85"} Jul 08 17:40:50.348313 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.348039 2580 scope.go:117] "RemoveContainer" containerID="e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab" Jul 08 17:40:50.357237 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.357216 2580 scope.go:117] "RemoveContainer" containerID="e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab" Jul 08 17:40:50.357486 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:50.357468 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab\": container with ID starting with e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab not found: ID does not exist" containerID="e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab" Jul 08 17:40:50.357528 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.357495 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab"} err="failed to get container status \"e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab\": rpc error: code = NotFound desc = could not find container \"e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab\": container with ID starting with e89790768f55555fff481c5e90097f6cc16059afe017b2a54a10b511d8cab2ab not found: ID does not exist" Jul 08 17:40:50.371607 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.371583 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d5d9b9b9-xhv7z"] Jul 08 17:40:50.375948 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:50.375926 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67d5d9b9b9-xhv7z"] Jul 08 17:40:51.845587 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:51.845541 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92bf3d37-03bc-4c12-ac59-ba433c62f4fc" path="/var/lib/kubelet/pods/92bf3d37-03bc-4c12-ac59-ba433c62f4fc/volumes" Jul 08 17:40:55.416482 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.416447 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68895d8676-sl25s" podUID="e4f5dde7-5633-413d-a8f6-0ab505c5faa1" containerName="console" containerID="cri-o://ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be" gracePeriod=15 Jul 08 17:40:55.686864 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.686761 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:55.743201 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743173 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-oauth-serving-cert\") pod \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " Jul 08 17:40:55.743363 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743212 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-trusted-ca-bundle\") pod \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " Jul 08 17:40:55.743363 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743348 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-service-ca\") pod \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " Jul 08 17:40:55.743497 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743391 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-serving-cert\") pod \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " Jul 08 17:40:55.743497 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743427 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb9ns\" (UniqueName: \"kubernetes.io/projected/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-kube-api-access-kb9ns\") pod \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " Jul 08 17:40:55.743497 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743451 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-config\") pod \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " Jul 08 17:40:55.743497 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743485 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-oauth-config\") pod \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\" (UID: \"e4f5dde7-5633-413d-a8f6-0ab505c5faa1\") " Jul 08 17:40:55.743710 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743685 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e4f5dde7-5633-413d-a8f6-0ab505c5faa1" (UID: "e4f5dde7-5633-413d-a8f6-0ab505c5faa1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:40:55.743763 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743721 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e4f5dde7-5633-413d-a8f6-0ab505c5faa1" (UID: "e4f5dde7-5633-413d-a8f6-0ab505c5faa1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:40:55.743816 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743751 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-service-ca" (OuterVolumeSpecName: "service-ca") pod "e4f5dde7-5633-413d-a8f6-0ab505c5faa1" (UID: "e4f5dde7-5633-413d-a8f6-0ab505c5faa1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:40:55.743917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743899 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-oauth-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:55.743964 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743921 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-trusted-ca-bundle\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:55.744005 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.743964 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-config" (OuterVolumeSpecName: "console-config") pod "e4f5dde7-5633-413d-a8f6-0ab505c5faa1" (UID: "e4f5dde7-5633-413d-a8f6-0ab505c5faa1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:40:55.745696 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.745670 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e4f5dde7-5633-413d-a8f6-0ab505c5faa1" (UID: "e4f5dde7-5633-413d-a8f6-0ab505c5faa1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:40:55.745794 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.745688 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-kube-api-access-kb9ns" (OuterVolumeSpecName: "kube-api-access-kb9ns") pod "e4f5dde7-5633-413d-a8f6-0ab505c5faa1" (UID: "e4f5dde7-5633-413d-a8f6-0ab505c5faa1"). InnerVolumeSpecName "kube-api-access-kb9ns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:40:55.745794 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.745713 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e4f5dde7-5633-413d-a8f6-0ab505c5faa1" (UID: "e4f5dde7-5633-413d-a8f6-0ab505c5faa1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:40:55.844610 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.844547 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-oauth-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:55.844610 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.844609 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-service-ca\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:55.844783 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.844619 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:55.844783 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.844629 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb9ns\" (UniqueName: \"kubernetes.io/projected/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-kube-api-access-kb9ns\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:55.844783 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:55.844639 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4f5dde7-5633-413d-a8f6-0ab505c5faa1-console-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:40:56.206284 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.206259 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ln4fx" Jul 08 17:40:56.374394 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.374357 2580 generic.go:356] "Generic (PLEG): container finished" podID="e4f5dde7-5633-413d-a8f6-0ab505c5faa1" containerID="ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be" exitCode=0 Jul 08 17:40:56.374545 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.374401 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68895d8676-sl25s" Jul 08 17:40:56.374545 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.374422 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68895d8676-sl25s" event={"ID":"e4f5dde7-5633-413d-a8f6-0ab505c5faa1","Type":"ContainerDied","Data":"ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be"} Jul 08 17:40:56.374545 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.374470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68895d8676-sl25s" event={"ID":"e4f5dde7-5633-413d-a8f6-0ab505c5faa1","Type":"ContainerDied","Data":"fd07b760faeda99237920beda73ad004633fd07a476a2b6c750ce021a022ca63"} Jul 08 17:40:56.374545 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.374496 2580 scope.go:117] "RemoveContainer" containerID="ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be" Jul 08 17:40:56.382812 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.382789 2580 scope.go:117] "RemoveContainer" containerID="ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be" Jul 08 17:40:56.383074 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:40:56.383053 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be\": container with ID starting with ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be not found: ID does not exist" containerID="ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be" Jul 08 17:40:56.383125 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.383083 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be"} err="failed to get container status \"ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be\": rpc error: code = NotFound desc = could not find container \"ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be\": container with ID starting with ebd70a13a0f177e33fae23e101ae331b818225c2f55ad2d4e0acdab7c4df17be not found: ID does not exist" Jul 08 17:40:56.393163 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.393141 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68895d8676-sl25s"] Jul 08 17:40:56.398651 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:56.398634 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68895d8676-sl25s"] Jul 08 17:40:57.845609 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:40:57.845557 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4f5dde7-5633-413d-a8f6-0ab505c5faa1" path="/var/lib/kubelet/pods/e4f5dde7-5633-413d-a8f6-0ab505c5faa1/volumes" Jul 08 17:41:03.081158 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:03.081128 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:41:03.085457 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:03.085429 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b59469cdf-v6z4k" Jul 08 17:41:06.409025 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.408962 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-686965cd8f-q6bw4" podUID="2fcc9853-95b2-4db6-88f2-0ab68f59a052" containerName="console" containerID="cri-o://089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6" gracePeriod=15 Jul 08 17:41:06.651307 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.651279 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:41:06.740890 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.740801 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kgzz\" (UniqueName: \"kubernetes.io/projected/2fcc9853-95b2-4db6-88f2-0ab68f59a052-kube-api-access-5kgzz\") pod \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " Jul 08 17:41:06.740890 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.740867 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-oauth-serving-cert\") pod \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " Jul 08 17:41:06.740890 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.740889 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-service-ca\") pod \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " Jul 08 17:41:06.741147 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741055 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-oauth-config\") pod \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " Jul 08 17:41:06.741201 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741145 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-trusted-ca-bundle\") pod \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " Jul 08 17:41:06.741201 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741152 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2fcc9853-95b2-4db6-88f2-0ab68f59a052" (UID: "2fcc9853-95b2-4db6-88f2-0ab68f59a052"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:41:06.741201 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741175 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-serving-cert\") pod \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " Jul 08 17:41:06.741333 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741204 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-config\") pod \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\" (UID: \"2fcc9853-95b2-4db6-88f2-0ab68f59a052\") " Jul 08 17:41:06.741333 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741228 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-service-ca" (OuterVolumeSpecName: "service-ca") pod "2fcc9853-95b2-4db6-88f2-0ab68f59a052" (UID: "2fcc9853-95b2-4db6-88f2-0ab68f59a052"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:41:06.741531 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741486 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-oauth-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:41:06.741531 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741495 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2fcc9853-95b2-4db6-88f2-0ab68f59a052" (UID: "2fcc9853-95b2-4db6-88f2-0ab68f59a052"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:41:06.741531 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741508 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-service-ca\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:41:06.741776 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.741662 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-config" (OuterVolumeSpecName: "console-config") pod "2fcc9853-95b2-4db6-88f2-0ab68f59a052" (UID: "2fcc9853-95b2-4db6-88f2-0ab68f59a052"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:41:06.743341 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.743317 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2fcc9853-95b2-4db6-88f2-0ab68f59a052" (UID: "2fcc9853-95b2-4db6-88f2-0ab68f59a052"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:41:06.743471 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.743452 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2fcc9853-95b2-4db6-88f2-0ab68f59a052" (UID: "2fcc9853-95b2-4db6-88f2-0ab68f59a052"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:41:06.743616 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.743595 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fcc9853-95b2-4db6-88f2-0ab68f59a052-kube-api-access-5kgzz" (OuterVolumeSpecName: "kube-api-access-5kgzz") pod "2fcc9853-95b2-4db6-88f2-0ab68f59a052" (UID: "2fcc9853-95b2-4db6-88f2-0ab68f59a052"). InnerVolumeSpecName "kube-api-access-5kgzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:41:06.842102 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.842070 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kgzz\" (UniqueName: \"kubernetes.io/projected/2fcc9853-95b2-4db6-88f2-0ab68f59a052-kube-api-access-5kgzz\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:41:06.842102 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.842099 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-oauth-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:41:06.842102 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.842108 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-trusted-ca-bundle\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:41:06.842102 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.842117 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:41:06.842362 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:06.842126 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2fcc9853-95b2-4db6-88f2-0ab68f59a052-console-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:41:07.423106 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.423069 2580 generic.go:356] "Generic (PLEG): container finished" podID="2fcc9853-95b2-4db6-88f2-0ab68f59a052" containerID="089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6" exitCode=0 Jul 08 17:41:07.423514 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.423116 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-686965cd8f-q6bw4" Jul 08 17:41:07.423514 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.423150 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-686965cd8f-q6bw4" event={"ID":"2fcc9853-95b2-4db6-88f2-0ab68f59a052","Type":"ContainerDied","Data":"089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6"} Jul 08 17:41:07.423514 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.423204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-686965cd8f-q6bw4" event={"ID":"2fcc9853-95b2-4db6-88f2-0ab68f59a052","Type":"ContainerDied","Data":"d7741a8777b0b49e3b7d30e9653257eff4d782131577e7162bd5f6f91c1106a6"} Jul 08 17:41:07.423514 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.423224 2580 scope.go:117] "RemoveContainer" containerID="089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6" Jul 08 17:41:07.432188 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.432172 2580 scope.go:117] "RemoveContainer" containerID="089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6" Jul 08 17:41:07.432438 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:41:07.432420 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6\": container with ID starting with 089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6 not found: ID does not exist" containerID="089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6" Jul 08 17:41:07.432486 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.432446 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6"} err="failed to get container status \"089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6\": rpc error: code = NotFound desc = could not find container \"089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6\": container with ID starting with 089936ada02f95d0293dd263a7cc9608b9ee1d7ff53d2b676237b5f9218508b6 not found: ID does not exist" Jul 08 17:41:07.447644 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.447621 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-686965cd8f-q6bw4"] Jul 08 17:41:07.452238 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.452220 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-686965cd8f-q6bw4"] Jul 08 17:41:07.845937 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:07.845906 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fcc9853-95b2-4db6-88f2-0ab68f59a052" path="/var/lib/kubelet/pods/2fcc9853-95b2-4db6-88f2-0ab68f59a052/volumes" Jul 08 17:41:25.137442 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:25.137407 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:41:25.158125 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:25.158097 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:41:25.517997 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:25.517922 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:41:52.943106 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943071 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bf4d8567c-vcjqp"] Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943392 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fcc9853-95b2-4db6-88f2-0ab68f59a052" containerName="console" Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943403 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fcc9853-95b2-4db6-88f2-0ab68f59a052" containerName="console" Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943415 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92bf3d37-03bc-4c12-ac59-ba433c62f4fc" containerName="console" Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943421 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="92bf3d37-03bc-4c12-ac59-ba433c62f4fc" containerName="console" Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943436 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f5dde7-5633-413d-a8f6-0ab505c5faa1" containerName="console" Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943441 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f5dde7-5633-413d-a8f6-0ab505c5faa1" containerName="console" Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943492 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="92bf3d37-03bc-4c12-ac59-ba433c62f4fc" containerName="console" Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943502 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fcc9853-95b2-4db6-88f2-0ab68f59a052" containerName="console" Jul 08 17:41:52.943589 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.943509 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4f5dde7-5633-413d-a8f6-0ab505c5faa1" containerName="console" Jul 08 17:41:52.945719 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.945700 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:52.957952 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:52.957903 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf4d8567c-vcjqp"] Jul 08 17:41:53.045458 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.045422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-oauth-config\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.045702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.045480 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-serving-cert\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.045702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.045604 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-config\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.045702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.045648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-trusted-ca-bundle\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.045702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.045678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hmll\" (UniqueName: \"kubernetes.io/projected/7d4ac4e4-497b-429d-bbe9-e034c5898571-kube-api-access-6hmll\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.045876 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.045722 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-service-ca\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.045876 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.045744 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-oauth-serving-cert\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.146563 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.146526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-service-ca\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.146563 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.146564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-oauth-serving-cert\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.146777 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.146643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-oauth-config\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.146777 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.146716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-serving-cert\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.146777 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.146765 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-config\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.146882 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.146788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-trusted-ca-bundle\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.146882 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.146818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmll\" (UniqueName: \"kubernetes.io/projected/7d4ac4e4-497b-429d-bbe9-e034c5898571-kube-api-access-6hmll\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.147375 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.147348 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-service-ca\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.147495 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.147373 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-oauth-serving-cert\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.147545 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.147511 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-config\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.148047 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.148021 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-trusted-ca-bundle\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.149172 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.149146 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-serving-cert\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.149379 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.149359 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-oauth-config\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.156672 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.156653 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hmll\" (UniqueName: \"kubernetes.io/projected/7d4ac4e4-497b-429d-bbe9-e034c5898571-kube-api-access-6hmll\") pod \"console-6bf4d8567c-vcjqp\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.254590 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.254486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:41:53.381014 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.380849 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf4d8567c-vcjqp"] Jul 08 17:41:53.383592 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:41:53.383554 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4ac4e4_497b_429d_bbe9_e034c5898571.slice/crio-c40bc909bf65b8510444595a6eef9befc8a572a7ea26bce0963f5744660c8f99 WatchSource:0}: Error finding container c40bc909bf65b8510444595a6eef9befc8a572a7ea26bce0963f5744660c8f99: Status 404 returned error can't find the container with id c40bc909bf65b8510444595a6eef9befc8a572a7ea26bce0963f5744660c8f99 Jul 08 17:41:53.610897 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.610864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf4d8567c-vcjqp" event={"ID":"7d4ac4e4-497b-429d-bbe9-e034c5898571","Type":"ContainerStarted","Data":"ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407"} Jul 08 17:41:53.611078 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.610911 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf4d8567c-vcjqp" event={"ID":"7d4ac4e4-497b-429d-bbe9-e034c5898571","Type":"ContainerStarted","Data":"c40bc909bf65b8510444595a6eef9befc8a572a7ea26bce0963f5744660c8f99"} Jul 08 17:41:53.633648 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:41:53.633594 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bf4d8567c-vcjqp" podStartSLOduration=1.63355983 podStartE2EDuration="1.63355983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:41:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:41:53.631833919 +0000 UTC m=+162.695356750" watchObservedRunningTime="2026-07-08 17:41:53.63355983 +0000 UTC m=+162.697082659" Jul 08 17:42:03.255305 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:03.255221 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:42:03.255305 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:03.255291 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:42:03.260081 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:03.260058 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:42:03.655399 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:03.655371 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:42:03.710252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:03.710219 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-597bcc946f-67dc5"] Jul 08 17:42:28.730564 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:28.730518 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-597bcc946f-67dc5" podUID="2e665281-fd4d-4226-bd97-409d08558ce7" containerName="console" containerID="cri-o://65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd" gracePeriod=15 Jul 08 17:42:28.977080 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:28.977057 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:42:29.057234 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057207 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-serving-cert\") pod \"2e665281-fd4d-4226-bd97-409d08558ce7\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " Jul 08 17:42:29.057411 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057270 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2k4c\" (UniqueName: \"kubernetes.io/projected/2e665281-fd4d-4226-bd97-409d08558ce7-kube-api-access-g2k4c\") pod \"2e665281-fd4d-4226-bd97-409d08558ce7\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " Jul 08 17:42:29.057411 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057289 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-oauth-config\") pod \"2e665281-fd4d-4226-bd97-409d08558ce7\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " Jul 08 17:42:29.057411 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057314 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-trusted-ca-bundle\") pod \"2e665281-fd4d-4226-bd97-409d08558ce7\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " Jul 08 17:42:29.057411 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057348 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-oauth-serving-cert\") pod \"2e665281-fd4d-4226-bd97-409d08558ce7\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " Jul 08 17:42:29.057411 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057369 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-console-config\") pod \"2e665281-fd4d-4226-bd97-409d08558ce7\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " Jul 08 17:42:29.057690 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057529 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-service-ca\") pod \"2e665281-fd4d-4226-bd97-409d08558ce7\" (UID: \"2e665281-fd4d-4226-bd97-409d08558ce7\") " Jul 08 17:42:29.057924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057788 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-console-config" (OuterVolumeSpecName: "console-config") pod "2e665281-fd4d-4226-bd97-409d08558ce7" (UID: "2e665281-fd4d-4226-bd97-409d08558ce7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:42:29.057924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057823 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2e665281-fd4d-4226-bd97-409d08558ce7" (UID: "2e665281-fd4d-4226-bd97-409d08558ce7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:42:29.057924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057865 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2e665281-fd4d-4226-bd97-409d08558ce7" (UID: "2e665281-fd4d-4226-bd97-409d08558ce7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:42:29.057924 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.057895 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-console-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:42:29.058246 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.058109 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-service-ca" (OuterVolumeSpecName: "service-ca") pod "2e665281-fd4d-4226-bd97-409d08558ce7" (UID: "2e665281-fd4d-4226-bd97-409d08558ce7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:42:29.059417 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.059392 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2e665281-fd4d-4226-bd97-409d08558ce7" (UID: "2e665281-fd4d-4226-bd97-409d08558ce7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:42:29.059554 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.059534 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2e665281-fd4d-4226-bd97-409d08558ce7" (UID: "2e665281-fd4d-4226-bd97-409d08558ce7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:42:29.059634 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.059602 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e665281-fd4d-4226-bd97-409d08558ce7-kube-api-access-g2k4c" (OuterVolumeSpecName: "kube-api-access-g2k4c") pod "2e665281-fd4d-4226-bd97-409d08558ce7" (UID: "2e665281-fd4d-4226-bd97-409d08558ce7"). InnerVolumeSpecName "kube-api-access-g2k4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:42:29.158556 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.158519 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2k4c\" (UniqueName: \"kubernetes.io/projected/2e665281-fd4d-4226-bd97-409d08558ce7-kube-api-access-g2k4c\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:42:29.158556 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.158550 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-oauth-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:42:29.158556 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.158560 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-trusted-ca-bundle\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:42:29.158809 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.158589 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-oauth-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:42:29.158809 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.158604 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e665281-fd4d-4226-bd97-409d08558ce7-service-ca\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:42:29.158809 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.158616 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e665281-fd4d-4226-bd97-409d08558ce7-console-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:42:29.763363 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.763333 2580 generic.go:356] "Generic (PLEG): container finished" podID="2e665281-fd4d-4226-bd97-409d08558ce7" containerID="65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd" exitCode=0 Jul 08 17:42:29.763779 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.763379 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597bcc946f-67dc5" Jul 08 17:42:29.763779 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.763399 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597bcc946f-67dc5" event={"ID":"2e665281-fd4d-4226-bd97-409d08558ce7","Type":"ContainerDied","Data":"65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd"} Jul 08 17:42:29.763779 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.763465 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597bcc946f-67dc5" event={"ID":"2e665281-fd4d-4226-bd97-409d08558ce7","Type":"ContainerDied","Data":"660d8b1fb871674304286f5d2f6852e0c5267672e29f92d186d92ed87fdf98b1"} Jul 08 17:42:29.763779 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.763485 2580 scope.go:117] "RemoveContainer" containerID="65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd" Jul 08 17:42:29.772251 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.772235 2580 scope.go:117] "RemoveContainer" containerID="65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd" Jul 08 17:42:29.772520 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:42:29.772503 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd\": container with ID starting with 65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd not found: ID does not exist" containerID="65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd" Jul 08 17:42:29.772564 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.772529 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd"} err="failed to get container status \"65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd\": rpc error: code = NotFound desc = could not find container \"65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd\": container with ID starting with 65058b1704ceb772e4cb2192e8c2c899b778a278509a22000f29bfaac9e322cd not found: ID does not exist" Jul 08 17:42:29.785126 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.785104 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-597bcc946f-67dc5"] Jul 08 17:42:29.789629 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.789606 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-597bcc946f-67dc5"] Jul 08 17:42:29.845412 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:42:29.845383 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e665281-fd4d-4226-bd97-409d08558ce7" path="/var/lib/kubelet/pods/2e665281-fd4d-4226-bd97-409d08558ce7/volumes" Jul 08 17:43:17.646394 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.646354 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk"] Jul 08 17:43:17.647025 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.646903 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e665281-fd4d-4226-bd97-409d08558ce7" containerName="console" Jul 08 17:43:17.647025 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.646924 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e665281-fd4d-4226-bd97-409d08558ce7" containerName="console" Jul 08 17:43:17.647158 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.647046 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e665281-fd4d-4226-bd97-409d08558ce7" containerName="console" Jul 08 17:43:17.649562 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.649542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" Jul 08 17:43:17.652056 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.652030 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 17:43:17.652147 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.652030 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-l62h4\"" Jul 08 17:43:17.652937 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.652925 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 17:43:17.659353 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.659331 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk"] Jul 08 17:43:17.778852 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.778805 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d408779-e4ad-41df-8647-3224e9c40ed3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vhtvk\" (UID: \"9d408779-e4ad-41df-8647-3224e9c40ed3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" Jul 08 17:43:17.779007 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.778931 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnjzv\" (UniqueName: \"kubernetes.io/projected/9d408779-e4ad-41df-8647-3224e9c40ed3-kube-api-access-xnjzv\") pod \"cert-manager-cainjector-559cdd8c87-vhtvk\" (UID: \"9d408779-e4ad-41df-8647-3224e9c40ed3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" Jul 08 17:43:17.879424 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.879385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnjzv\" (UniqueName: \"kubernetes.io/projected/9d408779-e4ad-41df-8647-3224e9c40ed3-kube-api-access-xnjzv\") pod \"cert-manager-cainjector-559cdd8c87-vhtvk\" (UID: \"9d408779-e4ad-41df-8647-3224e9c40ed3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" Jul 08 17:43:17.879631 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.879487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d408779-e4ad-41df-8647-3224e9c40ed3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vhtvk\" (UID: \"9d408779-e4ad-41df-8647-3224e9c40ed3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" Jul 08 17:43:17.891588 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.891537 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnjzv\" (UniqueName: \"kubernetes.io/projected/9d408779-e4ad-41df-8647-3224e9c40ed3-kube-api-access-xnjzv\") pod \"cert-manager-cainjector-559cdd8c87-vhtvk\" (UID: \"9d408779-e4ad-41df-8647-3224e9c40ed3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" Jul 08 17:43:17.891988 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.891970 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d408779-e4ad-41df-8647-3224e9c40ed3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vhtvk\" (UID: \"9d408779-e4ad-41df-8647-3224e9c40ed3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" Jul 08 17:43:17.970351 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:17.970277 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" Jul 08 17:43:18.097502 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:18.097475 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk"] Jul 08 17:43:18.099288 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:43:18.099261 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d408779_e4ad_41df_8647_3224e9c40ed3.slice/crio-84919ea72171389677df23c48fcce1f063ea99276d4e942594eb0f2aaaa41039 WatchSource:0}: Error finding container 84919ea72171389677df23c48fcce1f063ea99276d4e942594eb0f2aaaa41039: Status 404 returned error can't find the container with id 84919ea72171389677df23c48fcce1f063ea99276d4e942594eb0f2aaaa41039 Jul 08 17:43:18.961745 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:18.961706 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" event={"ID":"9d408779-e4ad-41df-8647-3224e9c40ed3","Type":"ContainerStarted","Data":"84919ea72171389677df23c48fcce1f063ea99276d4e942594eb0f2aaaa41039"} Jul 08 17:43:21.976837 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:21.976796 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" event={"ID":"9d408779-e4ad-41df-8647-3224e9c40ed3","Type":"ContainerStarted","Data":"48bd4bd98639f38ed51b9e398fd1204e1de47a4214b4590824a05e9bc972f426"} Jul 08 17:43:21.998867 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:21.998815 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vhtvk" podStartSLOduration=1.881067484 podStartE2EDuration="4.998800365s" totalImagesPullingTime="3.117732881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:43:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:43:21.99777251 +0000 UTC m=+251.061295342" watchObservedRunningTime="2026-07-08 17:43:21.998800365 +0000 UTC m=+251.062323194" Jul 08 17:43:24.366410 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.366378 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tgv8h"] Jul 08 17:43:24.368887 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.368866 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tgv8h" Jul 08 17:43:24.370982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.370965 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-q82hg\"" Jul 08 17:43:24.379067 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.379049 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tgv8h"] Jul 08 17:43:24.435798 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.435773 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5p4p\" (UniqueName: \"kubernetes.io/projected/57c262b8-8403-4b5a-b2e6-60773f46cd99-kube-api-access-g5p4p\") pod \"cert-manager-55b4d954d5-tgv8h\" (UID: \"57c262b8-8403-4b5a-b2e6-60773f46cd99\") " pod="cert-manager/cert-manager-55b4d954d5-tgv8h" Jul 08 17:43:24.435913 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.435804 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/57c262b8-8403-4b5a-b2e6-60773f46cd99-bound-sa-token\") pod \"cert-manager-55b4d954d5-tgv8h\" (UID: \"57c262b8-8403-4b5a-b2e6-60773f46cd99\") " pod="cert-manager/cert-manager-55b4d954d5-tgv8h" Jul 08 17:43:24.537384 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.537352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5p4p\" (UniqueName: \"kubernetes.io/projected/57c262b8-8403-4b5a-b2e6-60773f46cd99-kube-api-access-g5p4p\") pod \"cert-manager-55b4d954d5-tgv8h\" (UID: \"57c262b8-8403-4b5a-b2e6-60773f46cd99\") " pod="cert-manager/cert-manager-55b4d954d5-tgv8h" Jul 08 17:43:24.537501 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.537391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/57c262b8-8403-4b5a-b2e6-60773f46cd99-bound-sa-token\") pod \"cert-manager-55b4d954d5-tgv8h\" (UID: \"57c262b8-8403-4b5a-b2e6-60773f46cd99\") " pod="cert-manager/cert-manager-55b4d954d5-tgv8h" Jul 08 17:43:24.547557 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.547523 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/57c262b8-8403-4b5a-b2e6-60773f46cd99-bound-sa-token\") pod \"cert-manager-55b4d954d5-tgv8h\" (UID: \"57c262b8-8403-4b5a-b2e6-60773f46cd99\") " pod="cert-manager/cert-manager-55b4d954d5-tgv8h" Jul 08 17:43:24.548289 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.548268 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5p4p\" (UniqueName: \"kubernetes.io/projected/57c262b8-8403-4b5a-b2e6-60773f46cd99-kube-api-access-g5p4p\") pod \"cert-manager-55b4d954d5-tgv8h\" (UID: \"57c262b8-8403-4b5a-b2e6-60773f46cd99\") " pod="cert-manager/cert-manager-55b4d954d5-tgv8h" Jul 08 17:43:24.677590 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.677477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tgv8h" Jul 08 17:43:24.803018 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.802994 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tgv8h"] Jul 08 17:43:24.805418 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:43:24.805397 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57c262b8_8403_4b5a_b2e6_60773f46cd99.slice/crio-233be254e5f61a2cd3df59a57425be9367bfd25c40e869de24c1f5a73b33b9c5 WatchSource:0}: Error finding container 233be254e5f61a2cd3df59a57425be9367bfd25c40e869de24c1f5a73b33b9c5: Status 404 returned error can't find the container with id 233be254e5f61a2cd3df59a57425be9367bfd25c40e869de24c1f5a73b33b9c5 Jul 08 17:43:24.990532 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.990450 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tgv8h" event={"ID":"57c262b8-8403-4b5a-b2e6-60773f46cd99","Type":"ContainerStarted","Data":"c99718008bec6676fa250ef00db8f65ff33e7d3f7f3920547ddf573e9695d909"} Jul 08 17:43:24.990532 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:24.990492 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tgv8h" event={"ID":"57c262b8-8403-4b5a-b2e6-60773f46cd99","Type":"ContainerStarted","Data":"233be254e5f61a2cd3df59a57425be9367bfd25c40e869de24c1f5a73b33b9c5"} Jul 08 17:43:25.010948 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:25.010900 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-tgv8h" podStartSLOduration=1.010883985 podStartE2EDuration="1.010883985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:43:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:43:25.010002996 +0000 UTC m=+254.073525819" watchObservedRunningTime="2026-07-08 17:43:25.010883985 +0000 UTC m=+254.074406813" Jul 08 17:43:51.032010 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.031980 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz"] Jul 08 17:43:51.035490 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.035471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.038677 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.038651 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 17:43:51.038795 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.038764 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 17:43:51.039463 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.039445 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:43:51.039543 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.039513 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 17:43:51.039752 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.039737 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4fllq\"" Jul 08 17:43:51.039935 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.039922 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 17:43:51.048642 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.048616 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz"] Jul 08 17:43:51.071678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.071651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/327f3329-448c-4183-9a12-1ae38cca041a-manager-config\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.071770 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.071699 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/327f3329-448c-4183-9a12-1ae38cca041a-cert\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.071837 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.071814 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/327f3329-448c-4183-9a12-1ae38cca041a-metrics-cert\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.071879 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.071845 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c48hp\" (UniqueName: \"kubernetes.io/projected/327f3329-448c-4183-9a12-1ae38cca041a-kube-api-access-c48hp\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.172716 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.172682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/327f3329-448c-4183-9a12-1ae38cca041a-metrics-cert\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.172716 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.172721 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c48hp\" (UniqueName: \"kubernetes.io/projected/327f3329-448c-4183-9a12-1ae38cca041a-kube-api-access-c48hp\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.172878 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.172775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/327f3329-448c-4183-9a12-1ae38cca041a-manager-config\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.172878 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.172802 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/327f3329-448c-4183-9a12-1ae38cca041a-cert\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.173461 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.173438 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/327f3329-448c-4183-9a12-1ae38cca041a-manager-config\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.175145 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.175119 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/327f3329-448c-4183-9a12-1ae38cca041a-metrics-cert\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.175226 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.175179 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/327f3329-448c-4183-9a12-1ae38cca041a-cert\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.185217 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.185192 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c48hp\" (UniqueName: \"kubernetes.io/projected/327f3329-448c-4183-9a12-1ae38cca041a-kube-api-access-c48hp\") pod \"lws-controller-manager-7fbcbf7958-ddhkz\" (UID: \"327f3329-448c-4183-9a12-1ae38cca041a\") " pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.345750 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.345720 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:51.497331 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:51.497296 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz"] Jul 08 17:43:51.501457 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:43:51.501425 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327f3329_448c_4183_9a12_1ae38cca041a.slice/crio-0d2c68cfb68fabd29e7a5f8e1580759dbaaeb328f1424be7090a7b37e8ee7701 WatchSource:0}: Error finding container 0d2c68cfb68fabd29e7a5f8e1580759dbaaeb328f1424be7090a7b37e8ee7701: Status 404 returned error can't find the container with id 0d2c68cfb68fabd29e7a5f8e1580759dbaaeb328f1424be7090a7b37e8ee7701 Jul 08 17:43:52.098785 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.098747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" event={"ID":"327f3329-448c-4183-9a12-1ae38cca041a","Type":"ContainerStarted","Data":"0d2c68cfb68fabd29e7a5f8e1580759dbaaeb328f1424be7090a7b37e8ee7701"} Jul 08 17:43:52.590635 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.590591 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq"] Jul 08 17:43:52.596023 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.595993 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.599095 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.599067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5jqb9\"" Jul 08 17:43:52.599220 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.599148 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 17:43:52.599481 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.599462 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 17:43:52.599597 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.599462 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 17:43:52.600172 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.600144 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 17:43:52.619529 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.619504 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq"] Jul 08 17:43:52.688203 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.688166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24cb335e-a12b-46ba-8f55-6abd43401e90-tmp\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.688390 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.688221 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24cb335e-a12b-46ba-8f55-6abd43401e90-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.688390 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.688275 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24cb335e-a12b-46ba-8f55-6abd43401e90-manifests\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.688390 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.688299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6m9h\" (UniqueName: \"kubernetes.io/projected/24cb335e-a12b-46ba-8f55-6abd43401e90-kube-api-access-c6m9h\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.688390 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.688351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24cb335e-a12b-46ba-8f55-6abd43401e90-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.788854 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.788821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24cb335e-a12b-46ba-8f55-6abd43401e90-tmp\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.789034 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.788868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24cb335e-a12b-46ba-8f55-6abd43401e90-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.789034 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.788895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24cb335e-a12b-46ba-8f55-6abd43401e90-manifests\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.789034 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.788922 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6m9h\" (UniqueName: \"kubernetes.io/projected/24cb335e-a12b-46ba-8f55-6abd43401e90-kube-api-access-c6m9h\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.789034 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.788955 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24cb335e-a12b-46ba-8f55-6abd43401e90-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.789393 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.789368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24cb335e-a12b-46ba-8f55-6abd43401e90-manifests\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.791334 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.791309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24cb335e-a12b-46ba-8f55-6abd43401e90-tmp\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.791829 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.791808 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24cb335e-a12b-46ba-8f55-6abd43401e90-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.791940 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.791811 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24cb335e-a12b-46ba-8f55-6abd43401e90-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.799515 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.799495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6m9h\" (UniqueName: \"kubernetes.io/projected/24cb335e-a12b-46ba-8f55-6abd43401e90-kube-api-access-c6m9h\") pod \"opendatahub-operator-controller-manager-6dcf78d495-69vcq\" (UID: \"24cb335e-a12b-46ba-8f55-6abd43401e90\") " pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:52.909941 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:52.909873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:43:54.218997 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:54.218942 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq"] Jul 08 17:43:54.221353 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:43:54.221323 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24cb335e_a12b_46ba_8f55_6abd43401e90.slice/crio-f0e7b9902f15e77690c7722cbae30d1916823840b4b7f575e113f5444b1a18de WatchSource:0}: Error finding container f0e7b9902f15e77690c7722cbae30d1916823840b4b7f575e113f5444b1a18de: Status 404 returned error can't find the container with id f0e7b9902f15e77690c7722cbae30d1916823840b4b7f575e113f5444b1a18de Jul 08 17:43:55.119065 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:55.118995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" event={"ID":"327f3329-448c-4183-9a12-1ae38cca041a","Type":"ContainerStarted","Data":"52f55db742cda24f7d3ed4a2e4ad582cd20d749b1a0123d3ff4e612c0d7ca210"} Jul 08 17:43:55.119253 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:55.119206 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:43:55.120907 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:55.120851 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" event={"ID":"24cb335e-a12b-46ba-8f55-6abd43401e90","Type":"ContainerStarted","Data":"f0e7b9902f15e77690c7722cbae30d1916823840b4b7f575e113f5444b1a18de"} Jul 08 17:43:55.141073 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:55.140170 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" podStartSLOduration=1.537959694 podStartE2EDuration="4.140149643s" totalImagesPullingTime="2.602189949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:43:55.139849531 +0000 UTC m=+284.203372360" watchObservedRunningTime="2026-07-08 17:43:55.140149643 +0000 UTC m=+284.203672473" Jul 08 17:43:57.131048 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:57.131006 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" event={"ID":"24cb335e-a12b-46ba-8f55-6abd43401e90","Type":"ContainerStarted","Data":"7d04fa1c201b6d5b38b34c9bd7cffb91b5bb2d655ede2198c32e40e84bce143b"} Jul 08 17:43:59.141297 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:59.141261 2580 generic.go:356] "Generic (PLEG): container finished" podID="24cb335e-a12b-46ba-8f55-6abd43401e90" containerID="7d04fa1c201b6d5b38b34c9bd7cffb91b5bb2d655ede2198c32e40e84bce143b" exitCode=0 Jul 08 17:43:59.141712 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:43:59.141334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" event={"ID":"24cb335e-a12b-46ba-8f55-6abd43401e90","Type":"ContainerDied","Data":"7d04fa1c201b6d5b38b34c9bd7cffb91b5bb2d655ede2198c32e40e84bce143b"} Jul 08 17:44:00.147596 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:00.147543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" event={"ID":"24cb335e-a12b-46ba-8f55-6abd43401e90","Type":"ContainerStarted","Data":"18e17cdc73ee905a1b597fea17ca91f431927189fe5dbf95d8f8e134dde091aa"} Jul 08 17:44:00.147963 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:00.147777 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:44:00.170182 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:00.170072 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" podStartSLOduration=3.546074575 podStartE2EDuration="8.170051586s" totalImagesPullingTime="3.562321553s" totalInitContainerRuntime="1.061655458s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:43:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:44:00.168274624 +0000 UTC m=+289.231797453" watchObservedRunningTime="2026-07-08 17:44:00.170051586 +0000 UTC m=+289.233574415" Jul 08 17:44:06.127986 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:06.127957 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7fbcbf7958-ddhkz" Jul 08 17:44:11.154851 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:11.154814 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dcf78d495-69vcq" Jul 08 17:44:11.382751 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:11.382718 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 17:44:13.519042 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.518989 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd"] Jul 08 17:44:13.522299 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.522284 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:13.525011 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.524990 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-c4jt6\"" Jul 08 17:44:13.525122 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.525023 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 17:44:13.525592 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.525557 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 17:44:13.525639 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.525627 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 17:44:13.525965 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.525947 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 17:44:13.534400 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.534380 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd"] Jul 08 17:44:13.587080 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.587045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tmp\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:13.587244 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.587097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tls-certs\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:13.587244 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.587181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59ns6\" (UniqueName: \"kubernetes.io/projected/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-kube-api-access-59ns6\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:13.688609 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.688558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59ns6\" (UniqueName: \"kubernetes.io/projected/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-kube-api-access-59ns6\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:13.688813 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.688783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tmp\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:13.688990 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.688969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tls-certs\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:13.689128 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:44:13.689106 2580 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 08 17:44:13.689213 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:44:13.689199 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tls-certs podName:f8a9e5ca-5edf-4afe-9ddb-b02b5f406152 nodeName:}" failed. No retries permitted until 2026-07-08 17:44:14.189176546 +0000 UTC m=+303.252699372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tls-certs") pod "kube-auth-proxy-64959b45fc-8nwtd" (UID: "f8a9e5ca-5edf-4afe-9ddb-b02b5f406152") : secret "kube-auth-proxy-tls" not found Jul 08 17:44:13.691112 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.691090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tmp\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:13.707212 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:13.707179 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59ns6\" (UniqueName: \"kubernetes.io/projected/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-kube-api-access-59ns6\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:14.194185 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:14.194151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tls-certs\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:14.196657 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:14.196633 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a9e5ca-5edf-4afe-9ddb-b02b5f406152-tls-certs\") pod \"kube-auth-proxy-64959b45fc-8nwtd\" (UID: \"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152\") " pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:14.431925 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:14.431894 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" Jul 08 17:44:14.586918 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:14.586892 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd"] Jul 08 17:44:14.589213 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:44:14.589187 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a9e5ca_5edf_4afe_9ddb_b02b5f406152.slice/crio-bde98bbbe2854cdb09c85eac8ef5f1ffe4b7f60094221f6a4cb673967a6c404a WatchSource:0}: Error finding container bde98bbbe2854cdb09c85eac8ef5f1ffe4b7f60094221f6a4cb673967a6c404a: Status 404 returned error can't find the container with id bde98bbbe2854cdb09c85eac8ef5f1ffe4b7f60094221f6a4cb673967a6c404a Jul 08 17:44:14.591302 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:14.591285 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:44:15.217453 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:15.217409 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" event={"ID":"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152","Type":"ContainerStarted","Data":"bde98bbbe2854cdb09c85eac8ef5f1ffe4b7f60094221f6a4cb673967a6c404a"} Jul 08 17:44:18.233128 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:18.233074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" event={"ID":"f8a9e5ca-5edf-4afe-9ddb-b02b5f406152","Type":"ContainerStarted","Data":"552e65ed18829bebafd20e5f2ff72e2fa83dff63e6d57637087f2fcf082e8eba"} Jul 08 17:44:18.253423 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:44:18.253376 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64959b45fc-8nwtd" podStartSLOduration=2.182805976 podStartE2EDuration="5.253360874s" totalImagesPullingTime="3.070554898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:44:18.252496521 +0000 UTC m=+307.316019349" watchObservedRunningTime="2026-07-08 17:44:18.253360874 +0000 UTC m=+307.316883702" Jul 08 17:45:33.960740 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:33.960711 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-575d77984-d2pw9"] Jul 08 17:45:33.963948 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:33.963930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:33.977621 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:33.977595 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-575d77984-d2pw9"] Jul 08 17:45:34.012424 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.012396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-oauth-config\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.012424 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.012423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-config\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.012596 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.012453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-trusted-ca-bundle\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.012596 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.012503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-serving-cert\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.012596 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.012587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q8m5\" (UniqueName: \"kubernetes.io/projected/0b3e1978-4c68-46c0-92ff-11bb8972c638-kube-api-access-6q8m5\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.012700 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.012634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-service-ca\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.012700 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.012673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-oauth-serving-cert\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114012 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.113987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-service-ca\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114143 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-oauth-serving-cert\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114143 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-oauth-config\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114143 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-config\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114143 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-trusted-ca-bundle\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114143 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-serving-cert\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114355 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114146 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q8m5\" (UniqueName: \"kubernetes.io/projected/0b3e1978-4c68-46c0-92ff-11bb8972c638-kube-api-access-6q8m5\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114877 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114850 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-service-ca\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114978 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114890 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-oauth-serving-cert\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.114978 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.114896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-config\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.115215 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.115192 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b3e1978-4c68-46c0-92ff-11bb8972c638-trusted-ca-bundle\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.116459 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.116437 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-serving-cert\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.116555 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.116537 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b3e1978-4c68-46c0-92ff-11bb8972c638-console-oauth-config\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.123969 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.123948 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q8m5\" (UniqueName: \"kubernetes.io/projected/0b3e1978-4c68-46c0-92ff-11bb8972c638-kube-api-access-6q8m5\") pod \"console-575d77984-d2pw9\" (UID: \"0b3e1978-4c68-46c0-92ff-11bb8972c638\") " pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.272869 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.272781 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:34.405834 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.405808 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-575d77984-d2pw9"] Jul 08 17:45:34.408230 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:45:34.408202 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b3e1978_4c68_46c0_92ff_11bb8972c638.slice/crio-bf84dd1350367c5f5ce98b570830caf771a66e16cf63f081c81f74da79a4cde5 WatchSource:0}: Error finding container bf84dd1350367c5f5ce98b570830caf771a66e16cf63f081c81f74da79a4cde5: Status 404 returned error can't find the container with id bf84dd1350367c5f5ce98b570830caf771a66e16cf63f081c81f74da79a4cde5 Jul 08 17:45:34.602006 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.601969 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-575d77984-d2pw9" event={"ID":"0b3e1978-4c68-46c0-92ff-11bb8972c638","Type":"ContainerStarted","Data":"8d802a3ae0c1a81232e112f91dab4bb5350ece71f0efc16816cdfee5ab9d99eb"} Jul 08 17:45:34.602206 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.602026 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-575d77984-d2pw9" event={"ID":"0b3e1978-4c68-46c0-92ff-11bb8972c638","Type":"ContainerStarted","Data":"bf84dd1350367c5f5ce98b570830caf771a66e16cf63f081c81f74da79a4cde5"} Jul 08 17:45:34.623448 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:34.623401 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-575d77984-d2pw9" podStartSLOduration=1.623385812 podStartE2EDuration="1.623385812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:45:34.621549677 +0000 UTC m=+383.685072504" watchObservedRunningTime="2026-07-08 17:45:34.623385812 +0000 UTC m=+383.686908642" Jul 08 17:45:44.273305 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:44.273267 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:44.273818 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:44.273356 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:44.278174 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:44.278151 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:44.648282 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:44.648255 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-575d77984-d2pw9" Jul 08 17:45:44.700028 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:45:44.699995 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bf4d8567c-vcjqp"] Jul 08 17:46:09.720688 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:09.720630 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6bf4d8567c-vcjqp" podUID="7d4ac4e4-497b-429d-bbe9-e034c5898571" containerName="console" containerID="cri-o://ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407" gracePeriod=15 Jul 08 17:46:09.969041 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:09.969018 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:46:10.031371 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031291 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-serving-cert\") pod \"7d4ac4e4-497b-429d-bbe9-e034c5898571\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " Jul 08 17:46:10.031371 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031331 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-trusted-ca-bundle\") pod \"7d4ac4e4-497b-429d-bbe9-e034c5898571\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " Jul 08 17:46:10.031371 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031348 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-oauth-serving-cert\") pod \"7d4ac4e4-497b-429d-bbe9-e034c5898571\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " Jul 08 17:46:10.031371 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031367 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-oauth-config\") pod \"7d4ac4e4-497b-429d-bbe9-e034c5898571\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " Jul 08 17:46:10.031668 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031388 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-config\") pod \"7d4ac4e4-497b-429d-bbe9-e034c5898571\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " Jul 08 17:46:10.031668 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031495 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hmll\" (UniqueName: \"kubernetes.io/projected/7d4ac4e4-497b-429d-bbe9-e034c5898571-kube-api-access-6hmll\") pod \"7d4ac4e4-497b-429d-bbe9-e034c5898571\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " Jul 08 17:46:10.031668 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031517 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-service-ca\") pod \"7d4ac4e4-497b-429d-bbe9-e034c5898571\" (UID: \"7d4ac4e4-497b-429d-bbe9-e034c5898571\") " Jul 08 17:46:10.031892 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031866 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7d4ac4e4-497b-429d-bbe9-e034c5898571" (UID: "7d4ac4e4-497b-429d-bbe9-e034c5898571"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:46:10.031892 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031880 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-config" (OuterVolumeSpecName: "console-config") pod "7d4ac4e4-497b-429d-bbe9-e034c5898571" (UID: "7d4ac4e4-497b-429d-bbe9-e034c5898571"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:46:10.032007 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.031935 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7d4ac4e4-497b-429d-bbe9-e034c5898571" (UID: "7d4ac4e4-497b-429d-bbe9-e034c5898571"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:46:10.032198 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.032166 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-service-ca" (OuterVolumeSpecName: "service-ca") pod "7d4ac4e4-497b-429d-bbe9-e034c5898571" (UID: "7d4ac4e4-497b-429d-bbe9-e034c5898571"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:46:10.033438 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.033413 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7d4ac4e4-497b-429d-bbe9-e034c5898571" (UID: "7d4ac4e4-497b-429d-bbe9-e034c5898571"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:46:10.033544 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.033487 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7d4ac4e4-497b-429d-bbe9-e034c5898571" (UID: "7d4ac4e4-497b-429d-bbe9-e034c5898571"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:46:10.033676 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.033655 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d4ac4e4-497b-429d-bbe9-e034c5898571-kube-api-access-6hmll" (OuterVolumeSpecName: "kube-api-access-6hmll") pod "7d4ac4e4-497b-429d-bbe9-e034c5898571" (UID: "7d4ac4e4-497b-429d-bbe9-e034c5898571"). InnerVolumeSpecName "kube-api-access-6hmll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:46:10.132829 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.132797 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hmll\" (UniqueName: \"kubernetes.io/projected/7d4ac4e4-497b-429d-bbe9-e034c5898571-kube-api-access-6hmll\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:10.132829 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.132826 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-service-ca\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:10.133004 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.132839 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:10.133004 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.132852 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-trusted-ca-bundle\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:10.133004 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.132865 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-oauth-serving-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:10.133004 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.132878 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-oauth-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:10.133004 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.132890 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d4ac4e4-497b-429d-bbe9-e034c5898571-console-config\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:10.762368 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.762335 2580 generic.go:356] "Generic (PLEG): container finished" podID="7d4ac4e4-497b-429d-bbe9-e034c5898571" containerID="ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407" exitCode=0 Jul 08 17:46:10.762798 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.762380 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf4d8567c-vcjqp" Jul 08 17:46:10.762798 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.762417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf4d8567c-vcjqp" event={"ID":"7d4ac4e4-497b-429d-bbe9-e034c5898571","Type":"ContainerDied","Data":"ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407"} Jul 08 17:46:10.762798 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.762465 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf4d8567c-vcjqp" event={"ID":"7d4ac4e4-497b-429d-bbe9-e034c5898571","Type":"ContainerDied","Data":"c40bc909bf65b8510444595a6eef9befc8a572a7ea26bce0963f5744660c8f99"} Jul 08 17:46:10.762798 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.762485 2580 scope.go:117] "RemoveContainer" containerID="ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407" Jul 08 17:46:10.776689 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.776670 2580 scope.go:117] "RemoveContainer" containerID="ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407" Jul 08 17:46:10.776946 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:46:10.776926 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407\": container with ID starting with ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407 not found: ID does not exist" containerID="ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407" Jul 08 17:46:10.777019 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.776953 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407"} err="failed to get container status \"ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407\": rpc error: code = NotFound desc = could not find container \"ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407\": container with ID starting with ef96012cc40c43b86f6879055c7951a163785c114dbc970b887eedfe5935c407 not found: ID does not exist" Jul 08 17:46:10.788368 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.788345 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bf4d8567c-vcjqp"] Jul 08 17:46:10.793066 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:10.793046 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bf4d8567c-vcjqp"] Jul 08 17:46:11.846616 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:11.846555 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d4ac4e4-497b-429d-bbe9-e034c5898571" path="/var/lib/kubelet/pods/7d4ac4e4-497b-429d-bbe9-e034c5898571/volumes" Jul 08 17:46:17.042812 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.042779 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck"] Jul 08 17:46:17.043261 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.043133 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d4ac4e4-497b-429d-bbe9-e034c5898571" containerName="console" Jul 08 17:46:17.043261 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.043143 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d4ac4e4-497b-429d-bbe9-e034c5898571" containerName="console" Jul 08 17:46:17.043261 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.043216 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d4ac4e4-497b-429d-bbe9-e034c5898571" containerName="console" Jul 08 17:46:17.047792 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.047775 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:17.050835 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.050812 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 17:46:17.050951 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.050825 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kqmbf\"" Jul 08 17:46:17.051086 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.051072 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 17:46:17.064764 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.064743 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck"] Jul 08 17:46:17.096723 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.096698 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25t8x\" (UniqueName: \"kubernetes.io/projected/b34ccd94-d9da-43c4-83d2-42360284d572-kube-api-access-25t8x\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" (UID: \"b34ccd94-d9da-43c4-83d2-42360284d572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:17.096834 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.096750 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34ccd94-d9da-43c4-83d2-42360284d572-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" (UID: \"b34ccd94-d9da-43c4-83d2-42360284d572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:17.198160 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.198131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25t8x\" (UniqueName: \"kubernetes.io/projected/b34ccd94-d9da-43c4-83d2-42360284d572-kube-api-access-25t8x\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" (UID: \"b34ccd94-d9da-43c4-83d2-42360284d572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:17.198277 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.198181 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34ccd94-d9da-43c4-83d2-42360284d572-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" (UID: \"b34ccd94-d9da-43c4-83d2-42360284d572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:17.198481 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.198466 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34ccd94-d9da-43c4-83d2-42360284d572-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" (UID: \"b34ccd94-d9da-43c4-83d2-42360284d572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:17.210581 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.210539 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25t8x\" (UniqueName: \"kubernetes.io/projected/b34ccd94-d9da-43c4-83d2-42360284d572-kube-api-access-25t8x\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" (UID: \"b34ccd94-d9da-43c4-83d2-42360284d572\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:17.357717 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.357695 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:17.500611 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.500583 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck"] Jul 08 17:46:17.502791 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:46:17.502758 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34ccd94_d9da_43c4_83d2_42360284d572.slice/crio-b00efefae6287501c26ca741aabc497c283d93265343a8c8639778734f0e9b2c WatchSource:0}: Error finding container b00efefae6287501c26ca741aabc497c283d93265343a8c8639778734f0e9b2c: Status 404 returned error can't find the container with id b00efefae6287501c26ca741aabc497c283d93265343a8c8639778734f0e9b2c Jul 08 17:46:17.724887 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.724807 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck"] Jul 08 17:46:17.729600 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.729561 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck"] Jul 08 17:46:17.749221 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.749194 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh"] Jul 08 17:46:17.753844 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.753824 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:17.766595 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.763737 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh"] Jul 08 17:46:17.804609 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.804558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7hqkh\" (UID: \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:17.804777 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.804737 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb9s5\" (UniqueName: \"kubernetes.io/projected/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-kube-api-access-cb9s5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7hqkh\" (UID: \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:17.906095 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.906046 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb9s5\" (UniqueName: \"kubernetes.io/projected/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-kube-api-access-cb9s5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7hqkh\" (UID: \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:17.906278 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.906140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7hqkh\" (UID: \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:17.906461 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.906446 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7hqkh\" (UID: \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:17.915477 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:17.915446 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb9s5\" (UniqueName: \"kubernetes.io/projected/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-kube-api-access-cb9s5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7hqkh\" (UID: \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:18.076437 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:18.076397 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:18.219492 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:18.219438 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh"] Jul 08 17:46:18.280022 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:46:18.279985 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd621cd1_8134_48b1_bf04_8a180ec9a7e8.slice/crio-e4452190cd5978da80f4f56ca9d453f6b2686dc1683480b88b453810344c583c WatchSource:0}: Error finding container e4452190cd5978da80f4f56ca9d453f6b2686dc1683480b88b453810344c583c: Status 404 returned error can't find the container with id e4452190cd5978da80f4f56ca9d453f6b2686dc1683480b88b453810344c583c Jul 08 17:46:18.805787 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:18.805756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" event={"ID":"cd621cd1-8134-48b1-bf04-8a180ec9a7e8","Type":"ContainerStarted","Data":"e4452190cd5978da80f4f56ca9d453f6b2686dc1683480b88b453810344c583c"} Jul 08 17:46:22.830237 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:22.830177 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" containerName="manager" containerID="cri-o://e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4" gracePeriod=2 Jul 08 17:46:22.832149 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:22.832112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" event={"ID":"cd621cd1-8134-48b1-bf04-8a180ec9a7e8","Type":"ContainerStarted","Data":"428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c"} Jul 08 17:46:22.832516 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:22.832173 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:22.832661 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:46:22.832637 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:22.854204 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:22.854156 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" podStartSLOduration=1.442392448 podStartE2EDuration="5.854141757s" totalImagesPullingTime="4.411749309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:46:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:46:22.852868473 +0000 UTC m=+431.916391301" watchObservedRunningTime="2026-07-08 17:46:22.854141757 +0000 UTC m=+431.917664791" Jul 08 17:46:22.854732 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:46:22.854702 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:23.085949 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.085925 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:23.088352 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:46:23.088327 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:23.162871 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.162841 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25t8x\" (UniqueName: \"kubernetes.io/projected/b34ccd94-d9da-43c4-83d2-42360284d572-kube-api-access-25t8x\") pod \"b34ccd94-d9da-43c4-83d2-42360284d572\" (UID: \"b34ccd94-d9da-43c4-83d2-42360284d572\") " Jul 08 17:46:23.163036 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.162916 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34ccd94-d9da-43c4-83d2-42360284d572-extensions-socket-volume\") pod \"b34ccd94-d9da-43c4-83d2-42360284d572\" (UID: \"b34ccd94-d9da-43c4-83d2-42360284d572\") " Jul 08 17:46:23.163164 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.163141 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34ccd94-d9da-43c4-83d2-42360284d572-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b34ccd94-d9da-43c4-83d2-42360284d572" (UID: "b34ccd94-d9da-43c4-83d2-42360284d572"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:46:23.164956 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.164933 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34ccd94-d9da-43c4-83d2-42360284d572-kube-api-access-25t8x" (OuterVolumeSpecName: "kube-api-access-25t8x") pod "b34ccd94-d9da-43c4-83d2-42360284d572" (UID: "b34ccd94-d9da-43c4-83d2-42360284d572"). InnerVolumeSpecName "kube-api-access-25t8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:46:23.264236 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.264163 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25t8x\" (UniqueName: \"kubernetes.io/projected/b34ccd94-d9da-43c4-83d2-42360284d572-kube-api-access-25t8x\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:23.264236 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.264189 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b34ccd94-d9da-43c4-83d2-42360284d572-extensions-socket-volume\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:23.838970 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.838936 2580 generic.go:356] "Generic (PLEG): container finished" podID="b34ccd94-d9da-43c4-83d2-42360284d572" containerID="e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4" exitCode=2 Jul 08 17:46:23.839413 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.838976 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:23.839413 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.839019 2580 scope.go:117] "RemoveContainer" containerID="e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4" Jul 08 17:46:23.841150 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:46:23.841123 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:23.846669 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.846525 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" path="/var/lib/kubelet/pods/b34ccd94-d9da-43c4-83d2-42360284d572/volumes" Jul 08 17:46:23.848787 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.848771 2580 scope.go:117] "RemoveContainer" containerID="e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4" Jul 08 17:46:23.849027 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:46:23.849007 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4\": container with ID starting with e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4 not found: ID does not exist" containerID="e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4" Jul 08 17:46:23.849092 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:23.849037 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4"} err="failed to get container status \"e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4\": rpc error: code = NotFound desc = could not find container \"e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4\": container with ID starting with e15e9ba6931abbf890be3df5c7a7021c94427a36e2b11f656cb1c576d5b95ac4 not found: ID does not exist" Jul 08 17:46:23.849150 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:46:23.849130 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vfnck\" is forbidden: User \"system:node:ip-10-0-129-217.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-217.ec2.internal' and this object" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vfnck" Jul 08 17:46:33.846264 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:33.846189 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:46.269160 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.269125 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh"] Jul 08 17:46:46.269618 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.269299 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" podUID="cd621cd1-8134-48b1-bf04-8a180ec9a7e8" containerName="manager" containerID="cri-o://428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c" gracePeriod=10 Jul 08 17:46:46.522428 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.522370 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:46.575117 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.575091 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-extensions-socket-volume\") pod \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\" (UID: \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\") " Jul 08 17:46:46.575262 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.575184 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb9s5\" (UniqueName: \"kubernetes.io/projected/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-kube-api-access-cb9s5\") pod \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\" (UID: \"cd621cd1-8134-48b1-bf04-8a180ec9a7e8\") " Jul 08 17:46:46.575632 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.575607 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cd621cd1-8134-48b1-bf04-8a180ec9a7e8" (UID: "cd621cd1-8134-48b1-bf04-8a180ec9a7e8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:46:46.577133 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.577109 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-kube-api-access-cb9s5" (OuterVolumeSpecName: "kube-api-access-cb9s5") pod "cd621cd1-8134-48b1-bf04-8a180ec9a7e8" (UID: "cd621cd1-8134-48b1-bf04-8a180ec9a7e8"). InnerVolumeSpecName "kube-api-access-cb9s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:46:46.676467 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.676436 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-extensions-socket-volume\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:46.676467 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.676464 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb9s5\" (UniqueName: \"kubernetes.io/projected/cd621cd1-8134-48b1-bf04-8a180ec9a7e8-kube-api-access-cb9s5\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:46:46.947400 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.947364 2580 generic.go:356] "Generic (PLEG): container finished" podID="cd621cd1-8134-48b1-bf04-8a180ec9a7e8" containerID="428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c" exitCode=0 Jul 08 17:46:46.947557 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.947411 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" Jul 08 17:46:46.947557 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.947449 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" event={"ID":"cd621cd1-8134-48b1-bf04-8a180ec9a7e8","Type":"ContainerDied","Data":"428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c"} Jul 08 17:46:46.947557 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.947504 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh" event={"ID":"cd621cd1-8134-48b1-bf04-8a180ec9a7e8","Type":"ContainerDied","Data":"e4452190cd5978da80f4f56ca9d453f6b2686dc1683480b88b453810344c583c"} Jul 08 17:46:46.947557 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.947527 2580 scope.go:117] "RemoveContainer" containerID="428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c" Jul 08 17:46:46.956297 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.956282 2580 scope.go:117] "RemoveContainer" containerID="428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c" Jul 08 17:46:46.956584 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:46:46.956551 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c\": container with ID starting with 428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c not found: ID does not exist" containerID="428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c" Jul 08 17:46:46.956661 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.956594 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c"} err="failed to get container status \"428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c\": rpc error: code = NotFound desc = could not find container \"428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c\": container with ID starting with 428f39639693467176a3fb5115720f1a9e1ae45c930012e124debb833182a92c not found: ID does not exist" Jul 08 17:46:46.969365 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.969337 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh"] Jul 08 17:46:46.975657 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:46.975637 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7hqkh"] Jul 08 17:46:47.846527 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:46:47.846492 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd621cd1-8134-48b1-bf04-8a180ec9a7e8" path="/var/lib/kubelet/pods/cd621cd1-8134-48b1-bf04-8a180ec9a7e8/volumes" Jul 08 17:47:05.971727 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.971692 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:47:05.972323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.972041 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd621cd1-8134-48b1-bf04-8a180ec9a7e8" containerName="manager" Jul 08 17:47:05.972323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.972051 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd621cd1-8134-48b1-bf04-8a180ec9a7e8" containerName="manager" Jul 08 17:47:05.972323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.972063 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" containerName="manager" Jul 08 17:47:05.972323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.972068 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" containerName="manager" Jul 08 17:47:05.972323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.972133 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd621cd1-8134-48b1-bf04-8a180ec9a7e8" containerName="manager" Jul 08 17:47:05.972323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.972145 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34ccd94-d9da-43c4-83d2-42360284d572" containerName="manager" Jul 08 17:47:05.973976 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.973960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:05.976469 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.976447 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wrtxr\"" Jul 08 17:47:05.977898 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.977883 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 17:47:05.977977 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.977943 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 17:47:05.978035 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.978028 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 17:47:05.983285 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:05.983262 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:47:06.022983 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.022951 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:47:06.053822 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.053791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b197173-e328-4a64-8a09-b4b69f6509e5-config-file\") pod \"limitador-limitador-54d6d988d5-qkswz\" (UID: \"5b197173-e328-4a64-8a09-b4b69f6509e5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:06.053962 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.053835 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9cxk\" (UniqueName: \"kubernetes.io/projected/5b197173-e328-4a64-8a09-b4b69f6509e5-kube-api-access-h9cxk\") pod \"limitador-limitador-54d6d988d5-qkswz\" (UID: \"5b197173-e328-4a64-8a09-b4b69f6509e5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:06.154817 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.154784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b197173-e328-4a64-8a09-b4b69f6509e5-config-file\") pod \"limitador-limitador-54d6d988d5-qkswz\" (UID: \"5b197173-e328-4a64-8a09-b4b69f6509e5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:06.154982 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.154832 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9cxk\" (UniqueName: \"kubernetes.io/projected/5b197173-e328-4a64-8a09-b4b69f6509e5-kube-api-access-h9cxk\") pod \"limitador-limitador-54d6d988d5-qkswz\" (UID: \"5b197173-e328-4a64-8a09-b4b69f6509e5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:06.155399 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.155380 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b197173-e328-4a64-8a09-b4b69f6509e5-config-file\") pod \"limitador-limitador-54d6d988d5-qkswz\" (UID: \"5b197173-e328-4a64-8a09-b4b69f6509e5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:06.164396 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.164372 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9cxk\" (UniqueName: \"kubernetes.io/projected/5b197173-e328-4a64-8a09-b4b69f6509e5-kube-api-access-h9cxk\") pod \"limitador-limitador-54d6d988d5-qkswz\" (UID: \"5b197173-e328-4a64-8a09-b4b69f6509e5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:06.284846 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.284760 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:06.421103 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.421051 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:47:06.423463 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:47:06.423437 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b197173_e328_4a64_8a09_b4b69f6509e5.slice/crio-7d59c5d1e75331e6b1ae76bc13f8851575815988d6296fbe4d2e4ca51bfea353 WatchSource:0}: Error finding container 7d59c5d1e75331e6b1ae76bc13f8851575815988d6296fbe4d2e4ca51bfea353: Status 404 returned error can't find the container with id 7d59c5d1e75331e6b1ae76bc13f8851575815988d6296fbe4d2e4ca51bfea353 Jul 08 17:47:06.618822 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.618791 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-njwz8"] Jul 08 17:47:06.622151 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.622134 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-njwz8" Jul 08 17:47:06.624733 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.624709 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mlx7t\"" Jul 08 17:47:06.630848 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.630808 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-njwz8"] Jul 08 17:47:06.760209 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.760176 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgwzm\" (UniqueName: \"kubernetes.io/projected/41560c52-1ada-4c42-b66c-f114cbf0eed4-kube-api-access-jgwzm\") pod \"authorino-75d556bd77-njwz8\" (UID: \"41560c52-1ada-4c42-b66c-f114cbf0eed4\") " pod="kuadrant-system/authorino-75d556bd77-njwz8" Jul 08 17:47:06.861662 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.861623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgwzm\" (UniqueName: \"kubernetes.io/projected/41560c52-1ada-4c42-b66c-f114cbf0eed4-kube-api-access-jgwzm\") pod \"authorino-75d556bd77-njwz8\" (UID: \"41560c52-1ada-4c42-b66c-f114cbf0eed4\") " pod="kuadrant-system/authorino-75d556bd77-njwz8" Jul 08 17:47:06.871753 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.871689 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgwzm\" (UniqueName: \"kubernetes.io/projected/41560c52-1ada-4c42-b66c-f114cbf0eed4-kube-api-access-jgwzm\") pod \"authorino-75d556bd77-njwz8\" (UID: \"41560c52-1ada-4c42-b66c-f114cbf0eed4\") " pod="kuadrant-system/authorino-75d556bd77-njwz8" Jul 08 17:47:06.932391 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:06.932352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-njwz8" Jul 08 17:47:07.050078 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:07.050006 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" event={"ID":"5b197173-e328-4a64-8a09-b4b69f6509e5","Type":"ContainerStarted","Data":"7d59c5d1e75331e6b1ae76bc13f8851575815988d6296fbe4d2e4ca51bfea353"} Jul 08 17:47:07.117720 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:07.117686 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-njwz8"] Jul 08 17:47:07.119319 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:47:07.119291 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41560c52_1ada_4c42_b66c_f114cbf0eed4.slice/crio-1b4386937b377bfb4a0bc58ab3afe9fdabb746f10bd57e9fc38a33ed0a98ad1e WatchSource:0}: Error finding container 1b4386937b377bfb4a0bc58ab3afe9fdabb746f10bd57e9fc38a33ed0a98ad1e: Status 404 returned error can't find the container with id 1b4386937b377bfb4a0bc58ab3afe9fdabb746f10bd57e9fc38a33ed0a98ad1e Jul 08 17:47:08.060386 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:08.060348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-njwz8" event={"ID":"41560c52-1ada-4c42-b66c-f114cbf0eed4","Type":"ContainerStarted","Data":"1b4386937b377bfb4a0bc58ab3afe9fdabb746f10bd57e9fc38a33ed0a98ad1e"} Jul 08 17:47:12.086446 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:12.086406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" event={"ID":"5b197173-e328-4a64-8a09-b4b69f6509e5","Type":"ContainerStarted","Data":"deeeb291a5d07eaa3ae6195b4afb48de90061f6e2ac1fb6547275dfe8e512e77"} Jul 08 17:47:12.086939 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:12.086482 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:12.087847 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:12.087817 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-njwz8" event={"ID":"41560c52-1ada-4c42-b66c-f114cbf0eed4","Type":"ContainerStarted","Data":"bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65"} Jul 08 17:47:12.116886 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:12.116836 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" podStartSLOduration=2.27107082 podStartE2EDuration="7.116823781s" totalImagesPullingTime="4.845752961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:47:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:47:12.114346147 +0000 UTC m=+481.177868974" watchObservedRunningTime="2026-07-08 17:47:12.116823781 +0000 UTC m=+481.180346609" Jul 08 17:47:23.093586 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:23.093547 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qkswz" Jul 08 17:47:23.113602 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:23.113532 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-njwz8" podStartSLOduration=12.963371982 podStartE2EDuration="17.11351617s" totalImagesPullingTime="4.150144188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:47:12.134676366 +0000 UTC m=+481.198199192" watchObservedRunningTime="2026-07-08 17:47:23.11351617 +0000 UTC m=+492.177038998" Jul 08 17:47:44.685755 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:44.685720 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-njwz8"] Jul 08 17:47:44.686563 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:44.685932 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-njwz8" podUID="41560c52-1ada-4c42-b66c-f114cbf0eed4" containerName="authorino" containerID="cri-o://bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65" gracePeriod=30 Jul 08 17:47:44.968202 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:44.968178 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-njwz8" Jul 08 17:47:45.113375 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.113335 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgwzm\" (UniqueName: \"kubernetes.io/projected/41560c52-1ada-4c42-b66c-f114cbf0eed4-kube-api-access-jgwzm\") pod \"41560c52-1ada-4c42-b66c-f114cbf0eed4\" (UID: \"41560c52-1ada-4c42-b66c-f114cbf0eed4\") " Jul 08 17:47:45.115533 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.115502 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41560c52-1ada-4c42-b66c-f114cbf0eed4-kube-api-access-jgwzm" (OuterVolumeSpecName: "kube-api-access-jgwzm") pod "41560c52-1ada-4c42-b66c-f114cbf0eed4" (UID: "41560c52-1ada-4c42-b66c-f114cbf0eed4"). InnerVolumeSpecName "kube-api-access-jgwzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:47:45.214462 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.214374 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgwzm\" (UniqueName: \"kubernetes.io/projected/41560c52-1ada-4c42-b66c-f114cbf0eed4-kube-api-access-jgwzm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:47:45.247448 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.247414 2580 generic.go:356] "Generic (PLEG): container finished" podID="41560c52-1ada-4c42-b66c-f114cbf0eed4" containerID="bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65" exitCode=0 Jul 08 17:47:45.247624 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.247457 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-njwz8" Jul 08 17:47:45.247624 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.247479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-njwz8" event={"ID":"41560c52-1ada-4c42-b66c-f114cbf0eed4","Type":"ContainerDied","Data":"bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65"} Jul 08 17:47:45.247624 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.247513 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-njwz8" event={"ID":"41560c52-1ada-4c42-b66c-f114cbf0eed4","Type":"ContainerDied","Data":"1b4386937b377bfb4a0bc58ab3afe9fdabb746f10bd57e9fc38a33ed0a98ad1e"} Jul 08 17:47:45.247624 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.247529 2580 scope.go:117] "RemoveContainer" containerID="bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65" Jul 08 17:47:45.256192 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.256174 2580 scope.go:117] "RemoveContainer" containerID="bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65" Jul 08 17:47:45.256484 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:47:45.256465 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65\": container with ID starting with bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65 not found: ID does not exist" containerID="bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65" Jul 08 17:47:45.256531 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.256494 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65"} err="failed to get container status \"bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65\": rpc error: code = NotFound desc = could not find container \"bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65\": container with ID starting with bccffd23f290961346009ba52fe8c4cbb77ed5505a6b58c5e6e89d4351f16f65 not found: ID does not exist" Jul 08 17:47:45.271675 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.271647 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-njwz8"] Jul 08 17:47:45.274085 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.274062 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-njwz8"] Jul 08 17:47:45.846980 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:45.846947 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41560c52-1ada-4c42-b66c-f114cbf0eed4" path="/var/lib/kubelet/pods/41560c52-1ada-4c42-b66c-f114cbf0eed4/volumes" Jul 08 17:47:49.078702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.078658 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76ffdf9b8f-bt55q"] Jul 08 17:47:49.079157 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.079142 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41560c52-1ada-4c42-b66c-f114cbf0eed4" containerName="authorino" Jul 08 17:47:49.079212 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.079159 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="41560c52-1ada-4c42-b66c-f114cbf0eed4" containerName="authorino" Jul 08 17:47:49.079260 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.079216 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="41560c52-1ada-4c42-b66c-f114cbf0eed4" containerName="authorino" Jul 08 17:47:49.082199 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.082179 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:49.084789 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.084771 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 17:47:49.084918 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.084779 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 17:47:49.084963 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.084924 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lqn5x\"" Jul 08 17:47:49.095876 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.095851 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76ffdf9b8f-bt55q"] Jul 08 17:47:49.151516 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.151480 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cskn2\" (UniqueName: \"kubernetes.io/projected/0ecd7287-0e83-4c41-810c-575241fee851-kube-api-access-cskn2\") pod \"maas-controller-76ffdf9b8f-bt55q\" (UID: \"0ecd7287-0e83-4c41-810c-575241fee851\") " pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:49.151516 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.151525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ecd7287-0e83-4c41-810c-575241fee851-webhook-cert\") pod \"maas-controller-76ffdf9b8f-bt55q\" (UID: \"0ecd7287-0e83-4c41-810c-575241fee851\") " pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:49.251343 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.251306 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b69778487-tg2bf"] Jul 08 17:47:49.252799 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.252778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cskn2\" (UniqueName: \"kubernetes.io/projected/0ecd7287-0e83-4c41-810c-575241fee851-kube-api-access-cskn2\") pod \"maas-controller-76ffdf9b8f-bt55q\" (UID: \"0ecd7287-0e83-4c41-810c-575241fee851\") " pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:49.252876 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.252820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ecd7287-0e83-4c41-810c-575241fee851-webhook-cert\") pod \"maas-controller-76ffdf9b8f-bt55q\" (UID: \"0ecd7287-0e83-4c41-810c-575241fee851\") " pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:49.253942 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.253927 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:49.255353 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.255336 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ecd7287-0e83-4c41-810c-575241fee851-webhook-cert\") pod \"maas-controller-76ffdf9b8f-bt55q\" (UID: \"0ecd7287-0e83-4c41-810c-575241fee851\") " pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:49.267295 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.267261 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b69778487-tg2bf"] Jul 08 17:47:49.274132 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.274103 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cskn2\" (UniqueName: \"kubernetes.io/projected/0ecd7287-0e83-4c41-810c-575241fee851-kube-api-access-cskn2\") pod \"maas-controller-76ffdf9b8f-bt55q\" (UID: \"0ecd7287-0e83-4c41-810c-575241fee851\") " pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:49.353412 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.353318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s67g8\" (UniqueName: \"kubernetes.io/projected/ee400856-056a-4ef5-8844-b4616aefa71a-kube-api-access-s67g8\") pod \"maas-controller-7b69778487-tg2bf\" (UID: \"ee400856-056a-4ef5-8844-b4616aefa71a\") " pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:49.353412 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.353398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee400856-056a-4ef5-8844-b4616aefa71a-webhook-cert\") pod \"maas-controller-7b69778487-tg2bf\" (UID: \"ee400856-056a-4ef5-8844-b4616aefa71a\") " pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:49.392363 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.392332 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:49.454749 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.454718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee400856-056a-4ef5-8844-b4616aefa71a-webhook-cert\") pod \"maas-controller-7b69778487-tg2bf\" (UID: \"ee400856-056a-4ef5-8844-b4616aefa71a\") " pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:49.454925 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.454810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s67g8\" (UniqueName: \"kubernetes.io/projected/ee400856-056a-4ef5-8844-b4616aefa71a-kube-api-access-s67g8\") pod \"maas-controller-7b69778487-tg2bf\" (UID: \"ee400856-056a-4ef5-8844-b4616aefa71a\") " pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:49.457953 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.457920 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee400856-056a-4ef5-8844-b4616aefa71a-webhook-cert\") pod \"maas-controller-7b69778487-tg2bf\" (UID: \"ee400856-056a-4ef5-8844-b4616aefa71a\") " pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:49.465062 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.465035 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s67g8\" (UniqueName: \"kubernetes.io/projected/ee400856-056a-4ef5-8844-b4616aefa71a-kube-api-access-s67g8\") pod \"maas-controller-7b69778487-tg2bf\" (UID: \"ee400856-056a-4ef5-8844-b4616aefa71a\") " pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:49.528016 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.527986 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76ffdf9b8f-bt55q"] Jul 08 17:47:49.529892 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:47:49.529867 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecd7287_0e83_4c41_810c_575241fee851.slice/crio-0dca0580db22583ce81aa46bf4423309f140b76e6f140191e3dfcd03ff2dfc65 WatchSource:0}: Error finding container 0dca0580db22583ce81aa46bf4423309f140b76e6f140191e3dfcd03ff2dfc65: Status 404 returned error can't find the container with id 0dca0580db22583ce81aa46bf4423309f140b76e6f140191e3dfcd03ff2dfc65 Jul 08 17:47:49.587372 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.587330 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:49.720719 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:49.720684 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b69778487-tg2bf"] Jul 08 17:47:49.724502 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:47:49.724476 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee400856_056a_4ef5_8844_b4616aefa71a.slice/crio-390a24d39051cb4850e692b884c152e1cfddb2073921bbeedbffab1b4ffaf9b6 WatchSource:0}: Error finding container 390a24d39051cb4850e692b884c152e1cfddb2073921bbeedbffab1b4ffaf9b6: Status 404 returned error can't find the container with id 390a24d39051cb4850e692b884c152e1cfddb2073921bbeedbffab1b4ffaf9b6 Jul 08 17:47:50.283484 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:50.283406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b69778487-tg2bf" event={"ID":"ee400856-056a-4ef5-8844-b4616aefa71a","Type":"ContainerStarted","Data":"390a24d39051cb4850e692b884c152e1cfddb2073921bbeedbffab1b4ffaf9b6"} Jul 08 17:47:50.285232 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:50.285194 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" event={"ID":"0ecd7287-0e83-4c41-810c-575241fee851","Type":"ContainerStarted","Data":"0dca0580db22583ce81aa46bf4423309f140b76e6f140191e3dfcd03ff2dfc65"} Jul 08 17:47:52.296511 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:52.296473 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b69778487-tg2bf" event={"ID":"ee400856-056a-4ef5-8844-b4616aefa71a","Type":"ContainerStarted","Data":"f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb"} Jul 08 17:47:52.296963 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:52.296603 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:47:52.298124 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:52.298094 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" event={"ID":"0ecd7287-0e83-4c41-810c-575241fee851","Type":"ContainerStarted","Data":"fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc"} Jul 08 17:47:52.298261 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:52.298178 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:47:52.316615 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:52.316551 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b69778487-tg2bf" podStartSLOduration=1.386318997 podStartE2EDuration="3.316532758s" totalImagesPullingTime="1.930213761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:47:52.314404484 +0000 UTC m=+521.377927326" watchObservedRunningTime="2026-07-08 17:47:52.316532758 +0000 UTC m=+521.380055587" Jul 08 17:47:52.332797 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:47:52.332749 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" podStartSLOduration=1.1975134299999999 podStartE2EDuration="3.33273815s" totalImagesPullingTime="2.13522472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:47:52.331413777 +0000 UTC m=+521.394936607" watchObservedRunningTime="2026-07-08 17:47:52.33273815 +0000 UTC m=+521.396260979" Jul 08 17:48:03.310870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.310793 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:48:03.310870 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.310859 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:48:03.380437 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.380389 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76ffdf9b8f-bt55q"] Jul 08 17:48:03.380712 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.380683 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" podUID="0ecd7287-0e83-4c41-810c-575241fee851" containerName="manager" containerID="cri-o://fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc" gracePeriod=30 Jul 08 17:48:03.626289 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.626267 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:48:03.688460 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.688430 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cskn2\" (UniqueName: \"kubernetes.io/projected/0ecd7287-0e83-4c41-810c-575241fee851-kube-api-access-cskn2\") pod \"0ecd7287-0e83-4c41-810c-575241fee851\" (UID: \"0ecd7287-0e83-4c41-810c-575241fee851\") " Jul 08 17:48:03.688668 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.688537 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ecd7287-0e83-4c41-810c-575241fee851-webhook-cert\") pod \"0ecd7287-0e83-4c41-810c-575241fee851\" (UID: \"0ecd7287-0e83-4c41-810c-575241fee851\") " Jul 08 17:48:03.690642 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.690615 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ecd7287-0e83-4c41-810c-575241fee851-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0ecd7287-0e83-4c41-810c-575241fee851" (UID: "0ecd7287-0e83-4c41-810c-575241fee851"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:48:03.690642 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.690626 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ecd7287-0e83-4c41-810c-575241fee851-kube-api-access-cskn2" (OuterVolumeSpecName: "kube-api-access-cskn2") pod "0ecd7287-0e83-4c41-810c-575241fee851" (UID: "0ecd7287-0e83-4c41-810c-575241fee851"). InnerVolumeSpecName "kube-api-access-cskn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:48:03.789686 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.789650 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ecd7287-0e83-4c41-810c-575241fee851-webhook-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:48:03.789686 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:03.789680 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cskn2\" (UniqueName: \"kubernetes.io/projected/0ecd7287-0e83-4c41-810c-575241fee851-kube-api-access-cskn2\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:48:04.357318 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.357279 2580 generic.go:356] "Generic (PLEG): container finished" podID="0ecd7287-0e83-4c41-810c-575241fee851" containerID="fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc" exitCode=0 Jul 08 17:48:04.357821 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.357330 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" Jul 08 17:48:04.357821 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.357366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" event={"ID":"0ecd7287-0e83-4c41-810c-575241fee851","Type":"ContainerDied","Data":"fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc"} Jul 08 17:48:04.357821 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.357423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76ffdf9b8f-bt55q" event={"ID":"0ecd7287-0e83-4c41-810c-575241fee851","Type":"ContainerDied","Data":"0dca0580db22583ce81aa46bf4423309f140b76e6f140191e3dfcd03ff2dfc65"} Jul 08 17:48:04.357821 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.357447 2580 scope.go:117] "RemoveContainer" containerID="fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc" Jul 08 17:48:04.365777 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.365756 2580 scope.go:117] "RemoveContainer" containerID="fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc" Jul 08 17:48:04.366038 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:48:04.366019 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc\": container with ID starting with fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc not found: ID does not exist" containerID="fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc" Jul 08 17:48:04.366112 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.366045 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc"} err="failed to get container status \"fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc\": rpc error: code = NotFound desc = could not find container \"fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc\": container with ID starting with fae82a79c48ae3ee87145a72f4eb92698c2e3b8a8ad768ac883756d20e4a5fcc not found: ID does not exist" Jul 08 17:48:04.375660 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.375637 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76ffdf9b8f-bt55q"] Jul 08 17:48:04.380147 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:04.380126 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76ffdf9b8f-bt55q"] Jul 08 17:48:05.847702 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:05.847672 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ecd7287-0e83-4c41-810c-575241fee851" path="/var/lib/kubelet/pods/0ecd7287-0e83-4c41-810c-575241fee851/volumes" Jul 08 17:48:26.020909 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.020872 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-72svh"] Jul 08 17:48:26.021295 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.021244 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ecd7287-0e83-4c41-810c-575241fee851" containerName="manager" Jul 08 17:48:26.021295 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.021254 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ecd7287-0e83-4c41-810c-575241fee851" containerName="manager" Jul 08 17:48:26.021372 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.021310 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ecd7287-0e83-4c41-810c-575241fee851" containerName="manager" Jul 08 17:48:26.027418 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.027401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:26.029879 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.029855 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7r5g5\"" Jul 08 17:48:26.029993 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.029854 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 17:48:26.035437 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.035410 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-72svh"] Jul 08 17:48:26.052201 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.052176 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-4trf4"] Jul 08 17:48:26.056599 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.056558 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:26.068214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.068190 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-4trf4"] Jul 08 17:48:26.093081 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.093057 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:48:26.097749 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.097728 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6jlc\" (UniqueName: \"kubernetes.io/projected/3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6-kube-api-access-n6jlc\") pod \"payload-pre-processing-7964457696-72svh\" (UID: \"3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:26.097838 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.097763 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-72svh\" (UID: \"3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:26.198885 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.198849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6jlc\" (UniqueName: \"kubernetes.io/projected/3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6-kube-api-access-n6jlc\") pod \"payload-pre-processing-7964457696-72svh\" (UID: \"3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:26.199070 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.198928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4633c308-1619-439d-a657-9a0a835309d3-plugins-config-volume\") pod \"payload-processing-58f86c7798-4trf4\" (UID: \"4633c308-1619-439d-a657-9a0a835309d3\") " pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:26.199070 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.198979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-72svh\" (UID: \"3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:26.199070 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.199037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdjxc\" (UniqueName: \"kubernetes.io/projected/4633c308-1619-439d-a657-9a0a835309d3-kube-api-access-fdjxc\") pod \"payload-processing-58f86c7798-4trf4\" (UID: \"4633c308-1619-439d-a657-9a0a835309d3\") " pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:26.199792 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.199772 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-72svh\" (UID: \"3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:26.213468 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.213448 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6jlc\" (UniqueName: \"kubernetes.io/projected/3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6-kube-api-access-n6jlc\") pod \"payload-pre-processing-7964457696-72svh\" (UID: \"3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:26.299962 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.299891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4633c308-1619-439d-a657-9a0a835309d3-plugins-config-volume\") pod \"payload-processing-58f86c7798-4trf4\" (UID: \"4633c308-1619-439d-a657-9a0a835309d3\") " pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:26.299962 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.299934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdjxc\" (UniqueName: \"kubernetes.io/projected/4633c308-1619-439d-a657-9a0a835309d3-kube-api-access-fdjxc\") pod \"payload-processing-58f86c7798-4trf4\" (UID: \"4633c308-1619-439d-a657-9a0a835309d3\") " pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:26.300614 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.300593 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4633c308-1619-439d-a657-9a0a835309d3-plugins-config-volume\") pod \"payload-processing-58f86c7798-4trf4\" (UID: \"4633c308-1619-439d-a657-9a0a835309d3\") " pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:26.309077 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.309056 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdjxc\" (UniqueName: \"kubernetes.io/projected/4633c308-1619-439d-a657-9a0a835309d3-kube-api-access-fdjxc\") pod \"payload-processing-58f86c7798-4trf4\" (UID: \"4633c308-1619-439d-a657-9a0a835309d3\") " pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:26.338869 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.338838 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:26.367052 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.367025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:26.487645 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.487560 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-72svh"] Jul 08 17:48:26.491292 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:48:26.491256 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3956ee0a_e1e8_42f9_a2d7_cb64ff9b63a6.slice/crio-4b8867261cc087644a3b44fb7eef1cd82c8e039b2e5d129eb392a72d2a00d17f WatchSource:0}: Error finding container 4b8867261cc087644a3b44fb7eef1cd82c8e039b2e5d129eb392a72d2a00d17f: Status 404 returned error can't find the container with id 4b8867261cc087644a3b44fb7eef1cd82c8e039b2e5d129eb392a72d2a00d17f Jul 08 17:48:26.516649 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:26.516626 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-4trf4"] Jul 08 17:48:26.519008 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:48:26.518984 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4633c308_1619_439d_a657_9a0a835309d3.slice/crio-185e9316f36b902aa28f43e1a20f292683a118945b5ecaba231072e20c2ff7d1 WatchSource:0}: Error finding container 185e9316f36b902aa28f43e1a20f292683a118945b5ecaba231072e20c2ff7d1: Status 404 returned error can't find the container with id 185e9316f36b902aa28f43e1a20f292683a118945b5ecaba231072e20c2ff7d1 Jul 08 17:48:27.477178 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:27.477127 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-72svh" event={"ID":"3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6","Type":"ContainerStarted","Data":"4b8867261cc087644a3b44fb7eef1cd82c8e039b2e5d129eb392a72d2a00d17f"} Jul 08 17:48:27.478666 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:27.478632 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-4trf4" event={"ID":"4633c308-1619-439d-a657-9a0a835309d3","Type":"ContainerStarted","Data":"185e9316f36b902aa28f43e1a20f292683a118945b5ecaba231072e20c2ff7d1"} Jul 08 17:48:28.485225 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:28.485186 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-72svh" event={"ID":"3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6","Type":"ContainerStarted","Data":"83fe686858cd18f682450524fb0b1c21b5fd36ae5f59156ea00e4fab18d71fb6"} Jul 08 17:48:28.485726 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:28.485467 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:28.487055 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:28.487032 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-4trf4" event={"ID":"4633c308-1619-439d-a657-9a0a835309d3","Type":"ContainerStarted","Data":"13b5d75ba6500c3f16435bbb8c81506c3193d96f471198a392b436fda00209f8"} Jul 08 17:48:28.487195 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:28.487078 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:28.503773 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:28.503731 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-72svh" podStartSLOduration=1.011426561 podStartE2EDuration="2.503717665s" totalImagesPullingTime="1.492291104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:48:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:48:28.502093896 +0000 UTC m=+557.565616749" watchObservedRunningTime="2026-07-08 17:48:28.503717665 +0000 UTC m=+557.567240493" Jul 08 17:48:28.519466 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:28.519414 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-4trf4" podStartSLOduration=1.05153184 podStartE2EDuration="2.51939646s" totalImagesPullingTime="1.46786462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:48:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:48:28.51888376 +0000 UTC m=+557.582406590" watchObservedRunningTime="2026-07-08 17:48:28.51939646 +0000 UTC m=+557.582919291" Jul 08 17:48:39.493430 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:39.493397 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-4trf4" Jul 08 17:48:39.494053 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:39.493478 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-72svh" Jul 08 17:48:50.896399 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:50.896362 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:48:57.693091 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:48:57.693058 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:49:13.990933 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:13.990904 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx"] Jul 08 17:49:13.994365 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:13.994342 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:13.997002 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:13.996980 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 17:49:13.997173 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:13.997161 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w5cfc\"" Jul 08 17:49:13.998509 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:13.998479 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 17:49:13.998645 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:13.998530 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 08 17:49:14.006628 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.006607 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx"] Jul 08 17:49:14.141108 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.141074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.141108 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.141108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzsgt\" (UniqueName: \"kubernetes.io/projected/3f205d52-3ba5-471c-8ad5-8e57912293d9-kube-api-access-rzsgt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.141364 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.141158 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.141364 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.141181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.141364 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.141274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.141364 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.141305 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f205d52-3ba5-471c-8ad5-8e57912293d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242241 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242241 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242434 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242434 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f205d52-3ba5-471c-8ad5-8e57912293d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242434 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242297 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242434 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzsgt\" (UniqueName: \"kubernetes.io/projected/3f205d52-3ba5-471c-8ad5-8e57912293d9-kube-api-access-rzsgt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242656 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242600 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242714 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242666 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.242714 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.242685 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.244602 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.244550 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f205d52-3ba5-471c-8ad5-8e57912293d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.244848 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.244831 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f205d52-3ba5-471c-8ad5-8e57912293d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.252859 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.252836 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzsgt\" (UniqueName: \"kubernetes.io/projected/3f205d52-3ba5-471c-8ad5-8e57912293d9-kube-api-access-rzsgt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lblcx\" (UID: \"3f205d52-3ba5-471c-8ad5-8e57912293d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.304607 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.304561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:14.434811 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.434771 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx"] Jul 08 17:49:14.437032 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:49:14.437005 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f205d52_3ba5_471c_8ad5_8e57912293d9.slice/crio-ecc47391aa20df6870a320afdad469d9965d5043404f2ca2f259e333659652ee WatchSource:0}: Error finding container ecc47391aa20df6870a320afdad469d9965d5043404f2ca2f259e333659652ee: Status 404 returned error can't find the container with id ecc47391aa20df6870a320afdad469d9965d5043404f2ca2f259e333659652ee Jul 08 17:49:14.708791 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:14.708752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" event={"ID":"3f205d52-3ba5-471c-8ad5-8e57912293d9","Type":"ContainerStarted","Data":"ecc47391aa20df6870a320afdad469d9965d5043404f2ca2f259e333659652ee"} Jul 08 17:49:15.193327 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:15.193287 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:49:15.714596 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:15.714495 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" event={"ID":"3f205d52-3ba5-471c-8ad5-8e57912293d9","Type":"ContainerStarted","Data":"dc518f89a47287f11bedb75530e5ae3f8edbe607150060b89f3901efe5ef8591"} Jul 08 17:49:15.735787 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:15.735735 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" podStartSLOduration=1.736442802 podStartE2EDuration="2.735719982s" totalImagesPullingTime="999.27718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:49:15.733996688 +0000 UTC m=+604.797519517" watchObservedRunningTime="2026-07-08 17:49:15.735719982 +0000 UTC m=+604.799242811" Jul 08 17:49:24.305525 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:24.305486 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:24.305980 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:24.305545 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:24.318276 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:24.318251 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:24.767071 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:24.766990 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lblcx" Jul 08 17:49:36.518011 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:36.517928 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:49:59.800056 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.800026 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz"] Jul 08 17:49:59.805782 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.805762 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:49:59.808817 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.808790 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 17:49:59.812501 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.812478 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz"] Jul 08 17:49:59.965411 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.965369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:49:59.965629 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.965455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:49:59.965629 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.965483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:49:59.965629 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.965514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:49:59.965629 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.965554 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:49:59.965780 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:49:59.965654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnv9x\" (UniqueName: \"kubernetes.io/projected/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-kube-api-access-qnv9x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066129 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066205 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066322 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnv9x\" (UniqueName: \"kubernetes.io/projected/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-kube-api-access-qnv9x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066508 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066729 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066705 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066843 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066758 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.066890 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.066853 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.068522 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.068501 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.068679 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.068662 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.076652 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.076633 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnv9x\" (UniqueName: \"kubernetes.io/projected/4e09c23f-b3c9-4b57-8192-d55e2eb488b3-kube-api-access-qnv9x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-q5xrz\" (UID: \"4e09c23f-b3c9-4b57-8192-d55e2eb488b3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.117475 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.117456 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:00.247637 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.247615 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz"] Jul 08 17:50:00.249173 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:50:00.249144 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e09c23f_b3c9_4b57_8192_d55e2eb488b3.slice/crio-fea8f0c66e2529a9b3d6ff58ab0396cd20c961a1b8adfe611ca829851dac0cff WatchSource:0}: Error finding container fea8f0c66e2529a9b3d6ff58ab0396cd20c961a1b8adfe611ca829851dac0cff: Status 404 returned error can't find the container with id fea8f0c66e2529a9b3d6ff58ab0396cd20c961a1b8adfe611ca829851dac0cff Jul 08 17:50:00.250654 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.250633 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:50:00.923454 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.923418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" event={"ID":"4e09c23f-b3c9-4b57-8192-d55e2eb488b3","Type":"ContainerStarted","Data":"12af01053d5d814a04a66799032edba819f17d372672530d745cb9eaa1ede06d"} Jul 08 17:50:00.923840 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.923470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" event={"ID":"4e09c23f-b3c9-4b57-8192-d55e2eb488b3","Type":"ContainerStarted","Data":"fea8f0c66e2529a9b3d6ff58ab0396cd20c961a1b8adfe611ca829851dac0cff"} Jul 08 17:50:00.943892 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:00.943846 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" podStartSLOduration=1.710675684 podStartE2EDuration="1.943831471s" totalImagesPullingTime="233.155787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:50:00.943477344 +0000 UTC m=+650.007000170" watchObservedRunningTime="2026-07-08 17:50:00.943831471 +0000 UTC m=+650.007354300" Jul 08 17:50:01.898376 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:01.898344 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:50:10.117829 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.117790 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:10.118214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.117854 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:10.130210 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.130179 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:10.554529 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.554497 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx"] Jul 08 17:50:10.558104 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.558080 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.561264 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.561246 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 17:50:10.563214 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.563187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.563303 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.563240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b0cce84-e2fd-431a-be7f-9dac9edee288-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.563303 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.563295 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.563397 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.563336 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzw68\" (UniqueName: \"kubernetes.io/projected/6b0cce84-e2fd-431a-be7f-9dac9edee288-kube-api-access-xzw68\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.563397 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.563385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.563492 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.563475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.570472 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.570448 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx"] Jul 08 17:50:10.664219 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.664412 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b0cce84-e2fd-431a-be7f-9dac9edee288-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.664412 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.664412 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664301 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzw68\" (UniqueName: \"kubernetes.io/projected/6b0cce84-e2fd-431a-be7f-9dac9edee288-kube-api-access-xzw68\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.664610 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.664610 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.664700 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664668 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.664840 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664819 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.664878 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.664848 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.666528 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.666505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b0cce84-e2fd-431a-be7f-9dac9edee288-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.666736 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.666716 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b0cce84-e2fd-431a-be7f-9dac9edee288-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.675132 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.675104 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzw68\" (UniqueName: \"kubernetes.io/projected/6b0cce84-e2fd-431a-be7f-9dac9edee288-kube-api-access-xzw68\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx\" (UID: \"6b0cce84-e2fd-431a-be7f-9dac9edee288\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.868056 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.867965 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:10.990335 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:10.990308 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-q5xrz" Jul 08 17:50:11.007071 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:11.007050 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx"] Jul 08 17:50:11.008936 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:50:11.008910 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b0cce84_e2fd_431a_be7f_9dac9edee288.slice/crio-522bc6889bc847272f1bf77ac7ada4ca62dedbdd96a349203371024d8398049c WatchSource:0}: Error finding container 522bc6889bc847272f1bf77ac7ada4ca62dedbdd96a349203371024d8398049c: Status 404 returned error can't find the container with id 522bc6889bc847272f1bf77ac7ada4ca62dedbdd96a349203371024d8398049c Jul 08 17:50:11.982865 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:11.982832 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" event={"ID":"6b0cce84-e2fd-431a-be7f-9dac9edee288","Type":"ContainerStarted","Data":"ee526012ec1d3b36cf26d37157b1b3c68000321105a6589bbe001a1cf68183d8"} Jul 08 17:50:11.983248 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:11.982874 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" event={"ID":"6b0cce84-e2fd-431a-be7f-9dac9edee288","Type":"ContainerStarted","Data":"522bc6889bc847272f1bf77ac7ada4ca62dedbdd96a349203371024d8398049c"} Jul 08 17:50:12.003422 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:12.003375 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" podStartSLOduration=1.806816215 podStartE2EDuration="2.003359447s" totalImagesPullingTime="196.543232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:50:12.001499477 +0000 UTC m=+661.065022306" watchObservedRunningTime="2026-07-08 17:50:12.003359447 +0000 UTC m=+661.066882276" Jul 08 17:50:13.190733 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:13.190700 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:50:20.868218 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:20.868176 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:20.868218 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:20.868229 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:20.880779 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:20.880758 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:50:21.038140 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:50:21.038112 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx" Jul 08 17:51:49.600957 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:51:49.600925 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:52:01.104798 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:01.104769 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:52:12.211197 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:12.211160 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:52:19.897169 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:19.897131 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:52:31.097752 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:31.097723 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:52:33.295079 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.294990 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b69778487-tg2bf"] Jul 08 17:52:33.295525 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.295205 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7b69778487-tg2bf" podUID="ee400856-056a-4ef5-8844-b4616aefa71a" containerName="manager" containerID="cri-o://f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb" gracePeriod=30 Jul 08 17:52:33.305332 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.305298 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7b69778487-tg2bf" podUID="ee400856-056a-4ef5-8844-b4616aefa71a" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.35:8081/readyz\": dial tcp 10.134.0.35:8081: connect: connection refused" Jul 08 17:52:33.541029 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.541003 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:52:33.661724 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.661688 2580 generic.go:356] "Generic (PLEG): container finished" podID="ee400856-056a-4ef5-8844-b4616aefa71a" containerID="f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb" exitCode=0 Jul 08 17:52:33.661917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.661733 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b69778487-tg2bf" Jul 08 17:52:33.661917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.661751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b69778487-tg2bf" event={"ID":"ee400856-056a-4ef5-8844-b4616aefa71a","Type":"ContainerDied","Data":"f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb"} Jul 08 17:52:33.661917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.661803 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b69778487-tg2bf" event={"ID":"ee400856-056a-4ef5-8844-b4616aefa71a","Type":"ContainerDied","Data":"390a24d39051cb4850e692b884c152e1cfddb2073921bbeedbffab1b4ffaf9b6"} Jul 08 17:52:33.661917 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.661832 2580 scope.go:117] "RemoveContainer" containerID="f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb" Jul 08 17:52:33.670032 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.670005 2580 scope.go:117] "RemoveContainer" containerID="f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb" Jul 08 17:52:33.670275 ip-10-0-129-217 kubenswrapper[2580]: E0708 17:52:33.670254 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb\": container with ID starting with f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb not found: ID does not exist" containerID="f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb" Jul 08 17:52:33.670338 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.670284 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb"} err="failed to get container status \"f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb\": rpc error: code = NotFound desc = could not find container \"f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb\": container with ID starting with f2bd1121dbdf7684b9c0e67a49a498d8f3e29fe4122fc01a583cfd92c82672cb not found: ID does not exist" Jul 08 17:52:33.673636 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.673619 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s67g8\" (UniqueName: \"kubernetes.io/projected/ee400856-056a-4ef5-8844-b4616aefa71a-kube-api-access-s67g8\") pod \"ee400856-056a-4ef5-8844-b4616aefa71a\" (UID: \"ee400856-056a-4ef5-8844-b4616aefa71a\") " Jul 08 17:52:33.673782 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.673767 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee400856-056a-4ef5-8844-b4616aefa71a-webhook-cert\") pod \"ee400856-056a-4ef5-8844-b4616aefa71a\" (UID: \"ee400856-056a-4ef5-8844-b4616aefa71a\") " Jul 08 17:52:33.675655 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.675630 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee400856-056a-4ef5-8844-b4616aefa71a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ee400856-056a-4ef5-8844-b4616aefa71a" (UID: "ee400856-056a-4ef5-8844-b4616aefa71a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:52:33.675745 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.675681 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee400856-056a-4ef5-8844-b4616aefa71a-kube-api-access-s67g8" (OuterVolumeSpecName: "kube-api-access-s67g8") pod "ee400856-056a-4ef5-8844-b4616aefa71a" (UID: "ee400856-056a-4ef5-8844-b4616aefa71a"). InnerVolumeSpecName "kube-api-access-s67g8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:52:33.775642 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.775613 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee400856-056a-4ef5-8844-b4616aefa71a-webhook-cert\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:52:33.775642 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.775636 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s67g8\" (UniqueName: \"kubernetes.io/projected/ee400856-056a-4ef5-8844-b4616aefa71a-kube-api-access-s67g8\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 17:52:33.978922 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.978852 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b69778487-tg2bf"] Jul 08 17:52:33.984004 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:33.983983 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b69778487-tg2bf"] Jul 08 17:52:34.730838 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.730810 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b69778487-j2kgn"] Jul 08 17:52:34.731479 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.731459 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee400856-056a-4ef5-8844-b4616aefa71a" containerName="manager" Jul 08 17:52:34.731479 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.731479 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee400856-056a-4ef5-8844-b4616aefa71a" containerName="manager" Jul 08 17:52:34.731678 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.731625 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee400856-056a-4ef5-8844-b4616aefa71a" containerName="manager" Jul 08 17:52:34.735992 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.735975 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:34.739620 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.739601 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 17:52:34.739737 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.739652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 17:52:34.739737 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.739652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lqn5x\"" Jul 08 17:52:34.745196 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.745176 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b69778487-j2kgn"] Jul 08 17:52:34.784273 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.784249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwhl6\" (UniqueName: \"kubernetes.io/projected/d465147e-20d0-45aa-8702-2ed592770dbe-kube-api-access-gwhl6\") pod \"maas-controller-7b69778487-j2kgn\" (UID: \"d465147e-20d0-45aa-8702-2ed592770dbe\") " pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:34.784425 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.784292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d465147e-20d0-45aa-8702-2ed592770dbe-webhook-cert\") pod \"maas-controller-7b69778487-j2kgn\" (UID: \"d465147e-20d0-45aa-8702-2ed592770dbe\") " pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:34.885806 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.885774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwhl6\" (UniqueName: \"kubernetes.io/projected/d465147e-20d0-45aa-8702-2ed592770dbe-kube-api-access-gwhl6\") pod \"maas-controller-7b69778487-j2kgn\" (UID: \"d465147e-20d0-45aa-8702-2ed592770dbe\") " pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:34.885933 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.885824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d465147e-20d0-45aa-8702-2ed592770dbe-webhook-cert\") pod \"maas-controller-7b69778487-j2kgn\" (UID: \"d465147e-20d0-45aa-8702-2ed592770dbe\") " pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:34.888256 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.888232 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d465147e-20d0-45aa-8702-2ed592770dbe-webhook-cert\") pod \"maas-controller-7b69778487-j2kgn\" (UID: \"d465147e-20d0-45aa-8702-2ed592770dbe\") " pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:34.895989 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:34.895966 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwhl6\" (UniqueName: \"kubernetes.io/projected/d465147e-20d0-45aa-8702-2ed592770dbe-kube-api-access-gwhl6\") pod \"maas-controller-7b69778487-j2kgn\" (UID: \"d465147e-20d0-45aa-8702-2ed592770dbe\") " pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:35.047898 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:35.047808 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:35.184538 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:35.184387 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b69778487-j2kgn"] Jul 08 17:52:35.187620 ip-10-0-129-217 kubenswrapper[2580]: W0708 17:52:35.187591 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd465147e_20d0_45aa_8702_2ed592770dbe.slice/crio-41989af02190b4ecfc85d194ce71e399177f4421e538824ebe8d1a1ed59a9ddc WatchSource:0}: Error finding container 41989af02190b4ecfc85d194ce71e399177f4421e538824ebe8d1a1ed59a9ddc: Status 404 returned error can't find the container with id 41989af02190b4ecfc85d194ce71e399177f4421e538824ebe8d1a1ed59a9ddc Jul 08 17:52:35.674717 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:35.674679 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b69778487-j2kgn" event={"ID":"d465147e-20d0-45aa-8702-2ed592770dbe","Type":"ContainerStarted","Data":"7de3365c79edd3a106ee2f9c915b333e32867a0456bc38bde9eb26043f11d496"} Jul 08 17:52:35.674874 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:35.674741 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b69778487-j2kgn" event={"ID":"d465147e-20d0-45aa-8702-2ed592770dbe","Type":"ContainerStarted","Data":"41989af02190b4ecfc85d194ce71e399177f4421e538824ebe8d1a1ed59a9ddc"} Jul 08 17:52:35.674874 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:35.674793 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:35.696323 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:35.696280 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b69778487-j2kgn" podStartSLOduration=1.302111524 podStartE2EDuration="1.696267052s" totalImagesPullingTime="394.155528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:52:35.694312101 +0000 UTC m=+804.757834929" watchObservedRunningTime="2026-07-08 17:52:35.696267052 +0000 UTC m=+804.759789881" Jul 08 17:52:35.847212 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:35.847133 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee400856-056a-4ef5-8844-b4616aefa71a" path="/var/lib/kubelet/pods/ee400856-056a-4ef5-8844-b4616aefa71a/volumes" Jul 08 17:52:46.686335 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:46.686306 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b69778487-j2kgn" Jul 08 17:52:58.090335 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:52:58.090296 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:53:10.897115 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:53:10.897083 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:53:45.790343 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:53:45.790309 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:53:55.094919 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:53:55.094885 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:54:05.206597 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:54:05.206508 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:54:10.901455 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:54:10.901422 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:54:22.597252 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:54:22.597213 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:54:32.194686 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:54:32.194656 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:54:44.689488 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:54:44.689459 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:54:54.298983 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:54:54.298951 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:55:19.991222 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:55:19.991185 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:55:30.689423 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:55:30.689386 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:55:54.392082 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:55:54.392044 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:56:07.102331 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:56:07.102296 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:56:22.699964 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:56:22.699928 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:56:32.695061 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:56:32.695026 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:56:42.590343 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:56:42.590306 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:56:52.098889 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:56:52.098853 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:57:01.907503 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:57:01.907469 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:58:47.805642 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:58:47.805603 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:58:57.196114 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:58:57.196075 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:59:09.898201 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:59:09.898167 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:59:21.402090 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:59:21.402056 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:59:32.791064 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:59:32.791024 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:59:44.302098 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:59:44.302065 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 17:59:52.502426 ip-10-0-129-217 kubenswrapper[2580]: I0708 17:59:52.502391 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:00:04.594970 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:00:04.594888 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:00:14.196974 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:00:14.196928 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:00:23.794905 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:00:23.794870 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:00:37.292292 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:00:37.292257 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:01:01.405706 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:01:01.405668 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:01:09.200800 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:01:09.200766 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:01:19.194823 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:01:19.194791 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:01:28.691195 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:01:28.691165 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:01:39.298606 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:01:39.298556 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:01:51.223343 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:01:51.223306 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:02:02.591281 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:02:02.591241 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:02:12.106959 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:02:12.106920 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:02:19.812101 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:02:19.812060 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:02:31.396159 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:02:31.396122 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:02:40.998526 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:02:40.998492 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:02:54.601382 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:02:54.601345 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:03:09.303732 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:03:09.303649 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:03:19.086104 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:03:19.086066 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:03:25.501121 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:03:25.501080 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:03:36.499631 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:03:36.499595 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:03:44.602940 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:03:44.602901 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:03:57.191040 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:03:57.191003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:04:09.601681 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:04:09.601646 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:04:22.297588 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:04:22.297533 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:04:31.497228 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:04:31.497189 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:04:43.793286 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:04:43.793249 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:04:52.305510 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:04:52.305472 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:05:16.112540 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:05:16.112495 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:05:27.390132 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:05:27.390047 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:05:36.914749 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:05:36.914714 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:05:48.010906 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:05:48.010867 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:07:10.899358 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:07:10.899325 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:07:20.111443 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:07:20.111410 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:07:36.321235 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:07:36.321198 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:07:44.329041 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:07:44.329003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:08:21.016808 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:21.016769 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:08:37.355233 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.355194 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f"] Jul 08 18:08:37.359014 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.358991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.363083 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.363063 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-40518b0a\"/\"openshift-service-ca.crt\"" Jul 08 18:08:37.364079 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.364058 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-40518b0a\"/\"default-dockercfg-6jnmf\"" Jul 08 18:08:37.364182 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.364067 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-40518b0a\"/\"kube-root-ca.crt\"" Jul 08 18:08:37.364182 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.364152 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-40518b0a\"/\"e2e-lifecycle-model-40518b0a-kserve-self-signed-certs\"" Jul 08 18:08:37.386836 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.386771 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f"] Jul 08 18:08:37.490714 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.490673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3bedad0-0f5a-490a-9632-b9b197dfc011-tls-certs\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.490714 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.490717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-dshm\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.490977 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.490803 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-home\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.490977 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.490830 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-model-cache\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.490977 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.490866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-tmp-dir\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.490977 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.490922 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g57xs\" (UniqueName: \"kubernetes.io/projected/d3bedad0-0f5a-490a-9632-b9b197dfc011-kube-api-access-g57xs\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.591723 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.591687 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-model-cache\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.591723 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.591732 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-tmp-dir\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.591994 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.591767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g57xs\" (UniqueName: \"kubernetes.io/projected/d3bedad0-0f5a-490a-9632-b9b197dfc011-kube-api-access-g57xs\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.591994 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.591838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3bedad0-0f5a-490a-9632-b9b197dfc011-tls-certs\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.591994 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.591856 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-dshm\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.592153 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.591989 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-home\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.592269 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.592156 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-tmp-dir\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.592269 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.592202 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-model-cache\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.592344 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.592264 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-home\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.594127 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.594106 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-dshm\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.594367 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.594351 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3bedad0-0f5a-490a-9632-b9b197dfc011-tls-certs\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.604733 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.604713 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g57xs\" (UniqueName: \"kubernetes.io/projected/d3bedad0-0f5a-490a-9632-b9b197dfc011-kube-api-access-g57xs\") pod \"e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.668839 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.668762 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:37.813420 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:37.813389 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f"] Jul 08 18:08:37.816644 ip-10-0-129-217 kubenswrapper[2580]: W0708 18:08:37.816614 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3bedad0_0f5a_490a_9632_b9b197dfc011.slice/crio-77aa4d25f1b7a646fb5fa79d31d73d3bb891d72d43ec8b785057218b89661f77 WatchSource:0}: Error finding container 77aa4d25f1b7a646fb5fa79d31d73d3bb891d72d43ec8b785057218b89661f77: Status 404 returned error can't find the container with id 77aa4d25f1b7a646fb5fa79d31d73d3bb891d72d43ec8b785057218b89661f77 Jul 08 18:08:38.361642 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:38.361608 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" event={"ID":"d3bedad0-0f5a-490a-9632-b9b197dfc011","Type":"ContainerStarted","Data":"a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c"} Jul 08 18:08:38.362031 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:38.361652 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" event={"ID":"d3bedad0-0f5a-490a-9632-b9b197dfc011","Type":"ContainerStarted","Data":"77aa4d25f1b7a646fb5fa79d31d73d3bb891d72d43ec8b785057218b89661f77"} Jul 08 18:08:38.408559 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:38.408510 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" podStartSLOduration=1.408492852 podStartE2EDuration="1.408492852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:08:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:08:38.407745006 +0000 UTC m=+1767.471267835" watchObservedRunningTime="2026-07-08 18:08:38.408492852 +0000 UTC m=+1767.472015684" Jul 08 18:08:40.427458 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:40.427422 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:08:47.669078 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:47.669040 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:47.669475 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:47.669089 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:47.681798 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:47.681775 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:08:48.424346 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:08:48.424308 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:09:00.106653 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:00.106609 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:09:10.812624 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:10.812588 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:09:16.500396 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:16.500359 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f"] Jul 08 18:09:21.514083 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:21.514046 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:09:31.521106 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.521066 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" podUID="d3bedad0-0f5a-490a-9632-b9b197dfc011" containerName="main" containerID="cri-o://a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c" gracePeriod=45 Jul 08 18:09:31.776966 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.776900 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:09:31.919721 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.919689 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3bedad0-0f5a-490a-9632-b9b197dfc011-tls-certs\") pod \"d3bedad0-0f5a-490a-9632-b9b197dfc011\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " Jul 08 18:09:31.919947 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.919748 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g57xs\" (UniqueName: \"kubernetes.io/projected/d3bedad0-0f5a-490a-9632-b9b197dfc011-kube-api-access-g57xs\") pod \"d3bedad0-0f5a-490a-9632-b9b197dfc011\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " Jul 08 18:09:31.919947 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.919799 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-tmp-dir\") pod \"d3bedad0-0f5a-490a-9632-b9b197dfc011\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " Jul 08 18:09:31.919947 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.919830 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-home\") pod \"d3bedad0-0f5a-490a-9632-b9b197dfc011\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " Jul 08 18:09:31.919947 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.919848 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-dshm\") pod \"d3bedad0-0f5a-490a-9632-b9b197dfc011\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " Jul 08 18:09:31.919947 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.919899 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-model-cache\") pod \"d3bedad0-0f5a-490a-9632-b9b197dfc011\" (UID: \"d3bedad0-0f5a-490a-9632-b9b197dfc011\") " Jul 08 18:09:31.920217 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.920104 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3bedad0-0f5a-490a-9632-b9b197dfc011" (UID: "d3bedad0-0f5a-490a-9632-b9b197dfc011"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:09:31.920217 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.920117 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-home" (OuterVolumeSpecName: "home") pod "d3bedad0-0f5a-490a-9632-b9b197dfc011" (UID: "d3bedad0-0f5a-490a-9632-b9b197dfc011"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:09:31.920293 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.920266 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3bedad0-0f5a-490a-9632-b9b197dfc011" (UID: "d3bedad0-0f5a-490a-9632-b9b197dfc011"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:09:31.920457 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.920434 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:09:31.920561 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.920460 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:09:31.920561 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.920474 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:09:31.922156 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.922120 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-dshm" (OuterVolumeSpecName: "dshm") pod "d3bedad0-0f5a-490a-9632-b9b197dfc011" (UID: "d3bedad0-0f5a-490a-9632-b9b197dfc011"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:09:31.922270 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.922185 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3bedad0-0f5a-490a-9632-b9b197dfc011-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3bedad0-0f5a-490a-9632-b9b197dfc011" (UID: "d3bedad0-0f5a-490a-9632-b9b197dfc011"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:09:31.922270 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:31.922209 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3bedad0-0f5a-490a-9632-b9b197dfc011-kube-api-access-g57xs" (OuterVolumeSpecName: "kube-api-access-g57xs") pod "d3bedad0-0f5a-490a-9632-b9b197dfc011" (UID: "d3bedad0-0f5a-490a-9632-b9b197dfc011"). InnerVolumeSpecName "kube-api-access-g57xs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:09:32.021712 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.021681 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3bedad0-0f5a-490a-9632-b9b197dfc011-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:09:32.021712 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.021710 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g57xs\" (UniqueName: \"kubernetes.io/projected/d3bedad0-0f5a-490a-9632-b9b197dfc011-kube-api-access-g57xs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:09:32.021712 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.021722 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3bedad0-0f5a-490a-9632-b9b197dfc011-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:09:32.631073 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.631040 2580 generic.go:356] "Generic (PLEG): container finished" podID="d3bedad0-0f5a-490a-9632-b9b197dfc011" containerID="a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c" exitCode=0 Jul 08 18:09:32.631504 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.631083 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" Jul 08 18:09:32.631504 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.631125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" event={"ID":"d3bedad0-0f5a-490a-9632-b9b197dfc011","Type":"ContainerDied","Data":"a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c"} Jul 08 18:09:32.631504 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.631170 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f" event={"ID":"d3bedad0-0f5a-490a-9632-b9b197dfc011","Type":"ContainerDied","Data":"77aa4d25f1b7a646fb5fa79d31d73d3bb891d72d43ec8b785057218b89661f77"} Jul 08 18:09:32.631504 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.631190 2580 scope.go:117] "RemoveContainer" containerID="a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c" Jul 08 18:09:32.643529 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.643506 2580 scope.go:117] "RemoveContainer" containerID="a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c" Jul 08 18:09:32.643944 ip-10-0-129-217 kubenswrapper[2580]: E0708 18:09:32.643918 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c\": container with ID starting with a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c not found: ID does not exist" containerID="a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c" Jul 08 18:09:32.644070 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.643954 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c"} err="failed to get container status \"a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c\": rpc error: code = NotFound desc = could not find container \"a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c\": container with ID starting with a13220cf30d4a9ee52ae9cdd917d74111402746598fea95dcb05ddddfedae19c not found: ID does not exist" Jul 08 18:09:32.661891 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.661856 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f"] Jul 08 18:09:32.663709 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:32.663685 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-40518b0a/e2e-lifecycle-model-40518b0a-kserve-6b7d558547-89j9f"] Jul 08 18:09:33.847040 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:33.847001 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3bedad0-0f5a-490a-9632-b9b197dfc011" path="/var/lib/kubelet/pods/d3bedad0-0f5a-490a-9632-b9b197dfc011/volumes" Jul 08 18:09:57.107429 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:09:57.107392 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:10:14.418648 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:10:14.418611 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:10:22.713785 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:10:22.713753 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:11:13.424910 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:13.424873 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:11:24.214220 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.214183 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6"] Jul 08 18:11:24.214694 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.214602 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3bedad0-0f5a-490a-9632-b9b197dfc011" containerName="main" Jul 08 18:11:24.214694 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.214614 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3bedad0-0f5a-490a-9632-b9b197dfc011" containerName="main" Jul 08 18:11:24.214694 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.214694 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3bedad0-0f5a-490a-9632-b9b197dfc011" containerName="main" Jul 08 18:11:24.217776 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.217757 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.220792 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.220773 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bd74c9\"/\"kube-root-ca.crt\"" Jul 08 18:11:24.222077 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.222053 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bd74c9\"/\"default-dockercfg-jh67d\"" Jul 08 18:11:24.222197 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.222053 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bd74c9\"/\"test-model-bd74c9-kserve-self-signed-certs\"" Jul 08 18:11:24.222263 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.222133 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bd74c9\"/\"openshift-service-ca.crt\"" Jul 08 18:11:24.232939 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.232915 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6"] Jul 08 18:11:24.325959 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.325923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-home\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.326152 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.325968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-model-cache\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.326152 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.326021 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/513443ac-a7c3-48b5-a22a-f38692bccdb6-tls-certs\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.326152 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.326048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q96wh\" (UniqueName: \"kubernetes.io/projected/513443ac-a7c3-48b5-a22a-f38692bccdb6-kube-api-access-q96wh\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.326152 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.326115 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-tmp-dir\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.326296 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.326198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-dshm\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.427466 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.427427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-model-cache\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.427692 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.427478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/513443ac-a7c3-48b5-a22a-f38692bccdb6-tls-certs\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.427692 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.427501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q96wh\" (UniqueName: \"kubernetes.io/projected/513443ac-a7c3-48b5-a22a-f38692bccdb6-kube-api-access-q96wh\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.427692 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.427524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-tmp-dir\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.427692 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.427617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-dshm\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.427916 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.427710 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-home\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.427964 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.427908 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-model-cache\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.427964 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.427942 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-tmp-dir\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.428032 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.428010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-home\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.429821 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.429801 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-dshm\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.430112 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.430095 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/513443ac-a7c3-48b5-a22a-f38692bccdb6-tls-certs\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.442113 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.442090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q96wh\" (UniqueName: \"kubernetes.io/projected/513443ac-a7c3-48b5-a22a-f38692bccdb6-kube-api-access-q96wh\") pod \"test-model-bd74c9-kserve-5cc75bf8dc-xdcq6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.526475 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.526386 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:24.679108 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.679082 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6"] Jul 08 18:11:24.681310 ip-10-0-129-217 kubenswrapper[2580]: W0708 18:11:24.681285 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod513443ac_a7c3_48b5_a22a_f38692bccdb6.slice/crio-8527c505f57fa920e6bd9f69a505db44555aa6c7dad043727d4ad3761a5080fb WatchSource:0}: Error finding container 8527c505f57fa920e6bd9f69a505db44555aa6c7dad043727d4ad3761a5080fb: Status 404 returned error can't find the container with id 8527c505f57fa920e6bd9f69a505db44555aa6c7dad043727d4ad3761a5080fb Jul 08 18:11:24.909644 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:24.909607 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:11:25.196184 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:25.196101 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" event={"ID":"513443ac-a7c3-48b5-a22a-f38692bccdb6","Type":"ContainerStarted","Data":"8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46"} Jul 08 18:11:25.196184 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:25.196143 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" event={"ID":"513443ac-a7c3-48b5-a22a-f38692bccdb6","Type":"ContainerStarted","Data":"8527c505f57fa920e6bd9f69a505db44555aa6c7dad043727d4ad3761a5080fb"} Jul 08 18:11:25.218084 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:25.218036 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" podStartSLOduration=1.218021007 podStartE2EDuration="1.218021007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:25.216740269 +0000 UTC m=+1934.280263101" watchObservedRunningTime="2026-07-08 18:11:25.218021007 +0000 UTC m=+1934.281543835" Jul 08 18:11:34.527279 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:34.527234 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:34.527724 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:34.527384 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:34.539962 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:34.539931 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:34.745269 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:34.745238 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:11:35.256508 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:35.256478 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:11:53.408640 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.408607 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t"] Jul 08 18:11:53.411732 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.411713 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.414663 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.414622 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-97d4b9\"/\"openshift-service-ca.crt\"" Jul 08 18:11:53.415618 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.415594 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-97d4b9\"/\"default-dockercfg-5xhbv\"" Jul 08 18:11:53.415618 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.415612 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-97d4b9\"/\"kube-root-ca.crt\"" Jul 08 18:11:53.415809 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.415595 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-97d4b9\"/\"test-model-97d4b9-kserve-self-signed-certs\"" Jul 08 18:11:53.427239 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.427217 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t"] Jul 08 18:11:53.523856 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.523820 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-model-cache\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.523856 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.523860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10fca1e5-6520-45c4-a57f-9bb6d692c497-tls-certs\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.524109 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.523878 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-home\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.524109 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.523964 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-dshm\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.524109 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.524005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqb5n\" (UniqueName: \"kubernetes.io/projected/10fca1e5-6520-45c4-a57f-9bb6d692c497-kube-api-access-zqb5n\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.524109 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.524102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-tmp-dir\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.624870 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.624833 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-dshm\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.624870 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.624879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqb5n\" (UniqueName: \"kubernetes.io/projected/10fca1e5-6520-45c4-a57f-9bb6d692c497-kube-api-access-zqb5n\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.625101 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.624929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-tmp-dir\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.625101 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.624994 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-model-cache\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.625101 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.625016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10fca1e5-6520-45c4-a57f-9bb6d692c497-tls-certs\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.625101 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.625031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-home\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.625370 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.625353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-home\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.625440 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.625417 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-tmp-dir\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.625499 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.625465 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-model-cache\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.627169 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.627151 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-dshm\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.627434 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.627419 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10fca1e5-6520-45c4-a57f-9bb6d692c497-tls-certs\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.638834 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.638812 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqb5n\" (UniqueName: \"kubernetes.io/projected/10fca1e5-6520-45c4-a57f-9bb6d692c497-kube-api-access-zqb5n\") pod \"test-model-97d4b9-kserve-6c66fd5f5f-t469t\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.722195 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.722107 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:11:53.865684 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:53.865658 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t"] Jul 08 18:11:53.867652 ip-10-0-129-217 kubenswrapper[2580]: W0708 18:11:53.867624 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10fca1e5_6520_45c4_a57f_9bb6d692c497.slice/crio-e6c484db655d93fc93f0b1c8e0e8654d9182dd867be6abacc1f1a933ce0d0978 WatchSource:0}: Error finding container e6c484db655d93fc93f0b1c8e0e8654d9182dd867be6abacc1f1a933ce0d0978: Status 404 returned error can't find the container with id e6c484db655d93fc93f0b1c8e0e8654d9182dd867be6abacc1f1a933ce0d0978 Jul 08 18:11:54.339267 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:54.339228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" event={"ID":"10fca1e5-6520-45c4-a57f-9bb6d692c497","Type":"ContainerStarted","Data":"ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490"} Jul 08 18:11:54.339442 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:54.339276 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" event={"ID":"10fca1e5-6520-45c4-a57f-9bb6d692c497","Type":"ContainerStarted","Data":"e6c484db655d93fc93f0b1c8e0e8654d9182dd867be6abacc1f1a933ce0d0978"} Jul 08 18:11:54.360918 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:54.360863 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" podStartSLOduration=1.3608493 podStartE2EDuration="1.3608493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:54.359077896 +0000 UTC m=+1963.422600727" watchObservedRunningTime="2026-07-08 18:11:54.3608493 +0000 UTC m=+1963.424372129" Jul 08 18:11:59.129839 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:11:59.129803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:12:03.722781 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:03.722683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:12:03.723245 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:03.722978 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:12:03.735647 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:03.735612 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:12:04.401243 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:04.401209 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:12:32.296169 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:32.296132 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6"] Jul 08 18:12:33.210876 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:33.210846 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t"] Jul 08 18:12:36.912676 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:36.912641 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:12:47.316812 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.316767 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" podUID="513443ac-a7c3-48b5-a22a-f38692bccdb6" containerName="main" containerID="cri-o://8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46" gracePeriod=45 Jul 08 18:12:47.569257 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.569193 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:12:47.613936 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.613904 2580 generic.go:356] "Generic (PLEG): container finished" podID="513443ac-a7c3-48b5-a22a-f38692bccdb6" containerID="8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46" exitCode=0 Jul 08 18:12:47.614102 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.613955 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" Jul 08 18:12:47.614102 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.613966 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" event={"ID":"513443ac-a7c3-48b5-a22a-f38692bccdb6","Type":"ContainerDied","Data":"8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46"} Jul 08 18:12:47.614102 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.614000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6" event={"ID":"513443ac-a7c3-48b5-a22a-f38692bccdb6","Type":"ContainerDied","Data":"8527c505f57fa920e6bd9f69a505db44555aa6c7dad043727d4ad3761a5080fb"} Jul 08 18:12:47.614102 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.614017 2580 scope.go:117] "RemoveContainer" containerID="8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46" Jul 08 18:12:47.623391 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.623364 2580 scope.go:117] "RemoveContainer" containerID="8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46" Jul 08 18:12:47.623713 ip-10-0-129-217 kubenswrapper[2580]: E0708 18:12:47.623684 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46\": container with ID starting with 8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46 not found: ID does not exist" containerID="8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46" Jul 08 18:12:47.623804 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.623722 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46"} err="failed to get container status \"8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46\": rpc error: code = NotFound desc = could not find container \"8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46\": container with ID starting with 8fb962622ebf58da5860cb4dd3b9d789122bc3189ad301a92128287e49b17c46 not found: ID does not exist" Jul 08 18:12:47.670687 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.670656 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-home\") pod \"513443ac-a7c3-48b5-a22a-f38692bccdb6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " Jul 08 18:12:47.670860 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.670699 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-model-cache\") pod \"513443ac-a7c3-48b5-a22a-f38692bccdb6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " Jul 08 18:12:47.670860 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.670730 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-tmp-dir\") pod \"513443ac-a7c3-48b5-a22a-f38692bccdb6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " Jul 08 18:12:47.670860 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.670807 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-dshm\") pod \"513443ac-a7c3-48b5-a22a-f38692bccdb6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " Jul 08 18:12:47.670860 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.670851 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/513443ac-a7c3-48b5-a22a-f38692bccdb6-tls-certs\") pod \"513443ac-a7c3-48b5-a22a-f38692bccdb6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " Jul 08 18:12:47.671060 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.670879 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q96wh\" (UniqueName: \"kubernetes.io/projected/513443ac-a7c3-48b5-a22a-f38692bccdb6-kube-api-access-q96wh\") pod \"513443ac-a7c3-48b5-a22a-f38692bccdb6\" (UID: \"513443ac-a7c3-48b5-a22a-f38692bccdb6\") " Jul 08 18:12:47.671060 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.671000 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-home" (OuterVolumeSpecName: "home") pod "513443ac-a7c3-48b5-a22a-f38692bccdb6" (UID: "513443ac-a7c3-48b5-a22a-f38692bccdb6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:47.671060 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.671013 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-model-cache" (OuterVolumeSpecName: "model-cache") pod "513443ac-a7c3-48b5-a22a-f38692bccdb6" (UID: "513443ac-a7c3-48b5-a22a-f38692bccdb6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:47.671060 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.671024 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "513443ac-a7c3-48b5-a22a-f38692bccdb6" (UID: "513443ac-a7c3-48b5-a22a-f38692bccdb6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:47.671262 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.671182 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:47.671262 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.671193 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:47.671262 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.671203 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:47.673168 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.673141 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/513443ac-a7c3-48b5-a22a-f38692bccdb6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "513443ac-a7c3-48b5-a22a-f38692bccdb6" (UID: "513443ac-a7c3-48b5-a22a-f38692bccdb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:12:47.673414 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.673396 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-dshm" (OuterVolumeSpecName: "dshm") pod "513443ac-a7c3-48b5-a22a-f38692bccdb6" (UID: "513443ac-a7c3-48b5-a22a-f38692bccdb6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:47.673461 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.673422 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/513443ac-a7c3-48b5-a22a-f38692bccdb6-kube-api-access-q96wh" (OuterVolumeSpecName: "kube-api-access-q96wh") pod "513443ac-a7c3-48b5-a22a-f38692bccdb6" (UID: "513443ac-a7c3-48b5-a22a-f38692bccdb6"). InnerVolumeSpecName "kube-api-access-q96wh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:12:47.772249 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.772216 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/513443ac-a7c3-48b5-a22a-f38692bccdb6-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:47.772249 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.772244 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/513443ac-a7c3-48b5-a22a-f38692bccdb6-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:47.772249 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.772256 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q96wh\" (UniqueName: \"kubernetes.io/projected/513443ac-a7c3-48b5-a22a-f38692bccdb6-kube-api-access-q96wh\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:47.946162 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.946126 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6"] Jul 08 18:12:47.955925 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:47.955897 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-bd74c9/test-model-bd74c9-kserve-5cc75bf8dc-xdcq6"] Jul 08 18:12:48.231472 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.231376 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" podUID="10fca1e5-6520-45c4-a57f-9bb6d692c497" containerName="main" containerID="cri-o://ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490" gracePeriod=45 Jul 08 18:12:48.489656 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.489593 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:12:48.579558 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.579519 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-dshm\") pod \"10fca1e5-6520-45c4-a57f-9bb6d692c497\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " Jul 08 18:12:48.579558 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.579559 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqb5n\" (UniqueName: \"kubernetes.io/projected/10fca1e5-6520-45c4-a57f-9bb6d692c497-kube-api-access-zqb5n\") pod \"10fca1e5-6520-45c4-a57f-9bb6d692c497\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " Jul 08 18:12:48.579803 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.579705 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10fca1e5-6520-45c4-a57f-9bb6d692c497-tls-certs\") pod \"10fca1e5-6520-45c4-a57f-9bb6d692c497\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " Jul 08 18:12:48.579803 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.579725 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-model-cache\") pod \"10fca1e5-6520-45c4-a57f-9bb6d692c497\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " Jul 08 18:12:48.579803 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.579742 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-home\") pod \"10fca1e5-6520-45c4-a57f-9bb6d692c497\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " Jul 08 18:12:48.579803 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.579796 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-tmp-dir\") pod \"10fca1e5-6520-45c4-a57f-9bb6d692c497\" (UID: \"10fca1e5-6520-45c4-a57f-9bb6d692c497\") " Jul 08 18:12:48.580152 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.580089 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-home" (OuterVolumeSpecName: "home") pod "10fca1e5-6520-45c4-a57f-9bb6d692c497" (UID: "10fca1e5-6520-45c4-a57f-9bb6d692c497"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:48.580152 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.580144 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-model-cache" (OuterVolumeSpecName: "model-cache") pod "10fca1e5-6520-45c4-a57f-9bb6d692c497" (UID: "10fca1e5-6520-45c4-a57f-9bb6d692c497"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:48.580296 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.580162 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "10fca1e5-6520-45c4-a57f-9bb6d692c497" (UID: "10fca1e5-6520-45c4-a57f-9bb6d692c497"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:48.581922 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.581892 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10fca1e5-6520-45c4-a57f-9bb6d692c497-kube-api-access-zqb5n" (OuterVolumeSpecName: "kube-api-access-zqb5n") pod "10fca1e5-6520-45c4-a57f-9bb6d692c497" (UID: "10fca1e5-6520-45c4-a57f-9bb6d692c497"). InnerVolumeSpecName "kube-api-access-zqb5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:12:48.582200 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.582178 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10fca1e5-6520-45c4-a57f-9bb6d692c497-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "10fca1e5-6520-45c4-a57f-9bb6d692c497" (UID: "10fca1e5-6520-45c4-a57f-9bb6d692c497"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:12:48.582302 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.582183 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-dshm" (OuterVolumeSpecName: "dshm") pod "10fca1e5-6520-45c4-a57f-9bb6d692c497" (UID: "10fca1e5-6520-45c4-a57f-9bb6d692c497"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:48.627722 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.627683 2580 generic.go:356] "Generic (PLEG): container finished" podID="10fca1e5-6520-45c4-a57f-9bb6d692c497" containerID="ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490" exitCode=0 Jul 08 18:12:48.627901 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.627836 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" event={"ID":"10fca1e5-6520-45c4-a57f-9bb6d692c497","Type":"ContainerDied","Data":"ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490"} Jul 08 18:12:48.627901 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.627869 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" event={"ID":"10fca1e5-6520-45c4-a57f-9bb6d692c497","Type":"ContainerDied","Data":"e6c484db655d93fc93f0b1c8e0e8654d9182dd867be6abacc1f1a933ce0d0978"} Jul 08 18:12:48.627901 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.627886 2580 scope.go:117] "RemoveContainer" containerID="ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490" Jul 08 18:12:48.628056 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.627904 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t" Jul 08 18:12:48.636845 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.636829 2580 scope.go:117] "RemoveContainer" containerID="ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490" Jul 08 18:12:48.637101 ip-10-0-129-217 kubenswrapper[2580]: E0708 18:12:48.637082 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490\": container with ID starting with ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490 not found: ID does not exist" containerID="ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490" Jul 08 18:12:48.637147 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.637109 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490"} err="failed to get container status \"ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490\": rpc error: code = NotFound desc = could not find container \"ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490\": container with ID starting with ce60033f9e96880e7dab3d35d5ff2126e86dbbf06a093541a02d42c884aca490 not found: ID does not exist" Jul 08 18:12:48.681143 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.681107 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10fca1e5-6520-45c4-a57f-9bb6d692c497-tls-certs\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:48.681143 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.681142 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-model-cache\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:48.681143 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.681153 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-home\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:48.681440 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.681185 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-tmp-dir\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:48.681440 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.681199 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10fca1e5-6520-45c4-a57f-9bb6d692c497-dshm\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:48.681440 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.681213 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqb5n\" (UniqueName: \"kubernetes.io/projected/10fca1e5-6520-45c4-a57f-9bb6d692c497-kube-api-access-zqb5n\") on node \"ip-10-0-129-217.ec2.internal\" DevicePath \"\"" Jul 08 18:12:48.684482 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.684448 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t"] Jul 08 18:12:48.690737 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:48.690709 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-97d4b9/test-model-97d4b9-kserve-6c66fd5f5f-t469t"] Jul 08 18:12:49.847027 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:49.846997 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10fca1e5-6520-45c4-a57f-9bb6d692c497" path="/var/lib/kubelet/pods/10fca1e5-6520-45c4-a57f-9bb6d692c497/volumes" Jul 08 18:12:49.847409 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:12:49.847396 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="513443ac-a7c3-48b5-a22a-f38692bccdb6" path="/var/lib/kubelet/pods/513443ac-a7c3-48b5-a22a-f38692bccdb6/volumes" Jul 08 18:13:21.501704 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:13:21.501666 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:13:35.792166 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:13:35.792080 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:13:54.099141 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:13:54.099107 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:14:05.102039 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:14:05.102003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:14:16.800267 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:14:16.800231 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:14:25.997798 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:14:25.997761 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:17:33.297606 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:33.297560 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:17:47.417298 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:47.417255 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qkswz"] Jul 08 18:17:50.484021 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:50.483992 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b69778487-j2kgn_d465147e-20d0-45aa-8702-2ed592770dbe/manager/0.log" Jul 08 18:17:50.987901 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:50.987865 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dcf78d495-69vcq_24cb335e-a12b-46ba-8f55-6abd43401e90/copy-manifests/0.log" Jul 08 18:17:50.996345 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:50.996319 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dcf78d495-69vcq_24cb335e-a12b-46ba-8f55-6abd43401e90/manager/0.log" Jul 08 18:17:53.148670 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:53.148634 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qkswz_5b197173-e328-4a64-8a09-b4b69f6509e5/limitador/0.log" Jul 08 18:17:53.832895 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:53.832862 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64959b45fc-8nwtd_f8a9e5ca-5edf-4afe-9ddb-b02b5f406152/kube-auth-proxy/0.log" Jul 08 18:17:54.177425 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:54.177328 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-72svh_3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6/payload-pre-processing/0.log" Jul 08 18:17:54.293623 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:54.293586 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-4trf4_4633c308-1619-439d-a657-9a0a835309d3/payload-processing/0.log" Jul 08 18:17:54.976437 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:54.976405 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lblcx_3f205d52-3ba5-471c-8ad5-8e57912293d9/main/0.log" Jul 08 18:17:55.194754 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:55.194725 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-q5xrz_4e09c23f-b3c9-4b57-8192-d55e2eb488b3/main/0.log" Jul 08 18:17:55.315275 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:17:55.315245 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-z7ltx_6b0cce84-e2fd-431a-be7f-9dac9edee288/main/0.log" Jul 08 18:18:02.818028 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:02.818003 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5jp9r_775aa56e-6a57-4992-8079-863793659229/global-pull-secret-syncer/0.log" Jul 08 18:18:03.088081 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:03.087978 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-slbgv_c964a201-501c-4f3b-91c1-df8d714fe4bf/konnectivity-agent/0.log" Jul 08 18:18:03.119094 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:03.119062 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-217.ec2.internal_1c857f80ddb13de0ede08c410a7a78ba/haproxy/0.log" Jul 08 18:18:09.160838 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:09.160801 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qkswz_5b197173-e328-4a64-8a09-b4b69f6509e5/limitador/0.log" Jul 08 18:18:10.987704 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:10.987659 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b558aba6-91c4-442a-9f38-3e7955cd2cd7/alertmanager/0.log" Jul 08 18:18:11.039140 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.039106 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b558aba6-91c4-442a-9f38-3e7955cd2cd7/config-reloader/0.log" Jul 08 18:18:11.071795 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.071771 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b558aba6-91c4-442a-9f38-3e7955cd2cd7/kube-rbac-proxy-web/0.log" Jul 08 18:18:11.106046 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.106029 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b558aba6-91c4-442a-9f38-3e7955cd2cd7/kube-rbac-proxy/0.log" Jul 08 18:18:11.157158 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.157134 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b558aba6-91c4-442a-9f38-3e7955cd2cd7/kube-rbac-proxy-metric/0.log" Jul 08 18:18:11.189037 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.189012 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b558aba6-91c4-442a-9f38-3e7955cd2cd7/prom-label-proxy/0.log" Jul 08 18:18:11.216834 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.216814 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b558aba6-91c4-442a-9f38-3e7955cd2cd7/init-config-reloader/0.log" Jul 08 18:18:11.310046 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.310009 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-67j7k_e16a73e0-7afc-4605-9412-2c1014bbb981/kube-state-metrics/0.log" Jul 08 18:18:11.334655 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.334636 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-67j7k_e16a73e0-7afc-4605-9412-2c1014bbb981/kube-rbac-proxy-main/0.log" Jul 08 18:18:11.369225 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.369202 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-67j7k_e16a73e0-7afc-4605-9412-2c1014bbb981/kube-rbac-proxy-self/0.log" Jul 08 18:18:11.408529 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.408509 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b59469cdf-v6z4k_bf2401c2-5f64-4cd1-8be4-3b1d7d694684/metrics-server/0.log" Jul 08 18:18:11.447051 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.447018 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-82zbw_a23c7252-c9bb-4b4c-8ac3-2c57da03b804/monitoring-plugin/0.log" Jul 08 18:18:11.488763 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.488731 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4pcsx_174183a0-be00-4cdf-a793-74b5af0b947c/node-exporter/0.log" Jul 08 18:18:11.532634 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.532610 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4pcsx_174183a0-be00-4cdf-a793-74b5af0b947c/kube-rbac-proxy/0.log" Jul 08 18:18:11.558440 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.558415 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4pcsx_174183a0-be00-4cdf-a793-74b5af0b947c/init-textfile/0.log" Jul 08 18:18:11.809479 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.809446 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qkfx9_31e1ec4a-1afc-4796-952f-92f875008da5/kube-rbac-proxy-main/0.log" Jul 08 18:18:11.839500 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.839474 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qkfx9_31e1ec4a-1afc-4796-952f-92f875008da5/kube-rbac-proxy-self/0.log" Jul 08 18:18:11.876038 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.876009 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qkfx9_31e1ec4a-1afc-4796-952f-92f875008da5/openshift-state-metrics/0.log" Jul 08 18:18:11.920386 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.920354 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3921707f-3ea9-4e71-b584-5777cd12edc4/prometheus/0.log" Jul 08 18:18:11.966582 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:11.966536 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3921707f-3ea9-4e71-b584-5777cd12edc4/config-reloader/0.log" Jul 08 18:18:12.002547 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.002521 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3921707f-3ea9-4e71-b584-5777cd12edc4/thanos-sidecar/0.log" Jul 08 18:18:12.031250 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.031230 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3921707f-3ea9-4e71-b584-5777cd12edc4/kube-rbac-proxy-web/0.log" Jul 08 18:18:12.073621 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.073528 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3921707f-3ea9-4e71-b584-5777cd12edc4/kube-rbac-proxy/0.log" Jul 08 18:18:12.107055 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.107024 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3921707f-3ea9-4e71-b584-5777cd12edc4/kube-rbac-proxy-thanos/0.log" Jul 08 18:18:12.136963 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.136939 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3921707f-3ea9-4e71-b584-5777cd12edc4/init-config-reloader/0.log" Jul 08 18:18:12.178295 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.178267 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-6znvt_09e2efdd-f874-480b-9f43-b4187123c3c3/prometheus-operator/0.log" Jul 08 18:18:12.209502 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.209479 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-6znvt_09e2efdd-f874-480b-9f43-b4187123c3c3/kube-rbac-proxy/0.log" Jul 08 18:18:12.250537 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.250511 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-kxrbl_c8231274-d8e9-4502-9312-808207d1dcf0/prometheus-operator-admission-webhook/0.log" Jul 08 18:18:12.290088 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.290059 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-575d485f8d-g8h49_236b6ca0-00c6-4ec3-803d-affd1b02c959/telemeter-client/0.log" Jul 08 18:18:12.325907 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.325827 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-575d485f8d-g8h49_236b6ca0-00c6-4ec3-803d-affd1b02c959/reload/0.log" Jul 08 18:18:12.360782 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.360756 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-575d485f8d-g8h49_236b6ca0-00c6-4ec3-803d-affd1b02c959/kube-rbac-proxy/0.log" Jul 08 18:18:12.407606 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.407561 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c5576cf-5xhcj_e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f/thanos-query/0.log" Jul 08 18:18:12.451817 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.451786 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c5576cf-5xhcj_e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f/kube-rbac-proxy-web/0.log" Jul 08 18:18:12.487493 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.487457 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c5576cf-5xhcj_e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f/kube-rbac-proxy/0.log" Jul 08 18:18:12.519457 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.519428 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c5576cf-5xhcj_e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f/prom-label-proxy/0.log" Jul 08 18:18:12.559638 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.559605 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c5576cf-5xhcj_e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f/kube-rbac-proxy-rules/0.log" Jul 08 18:18:12.587130 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.587057 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c6c5576cf-5xhcj_e2fa2c47-aad4-4f89-a85f-a6be5b4c8f3f/kube-rbac-proxy-metrics/0.log" Jul 08 18:18:12.685100 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.685069 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss"] Jul 08 18:18:12.685474 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.685463 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10fca1e5-6520-45c4-a57f-9bb6d692c497" containerName="main" Jul 08 18:18:12.685526 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.685476 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10fca1e5-6520-45c4-a57f-9bb6d692c497" containerName="main" Jul 08 18:18:12.685526 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.685488 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="513443ac-a7c3-48b5-a22a-f38692bccdb6" containerName="main" Jul 08 18:18:12.685526 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.685493 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="513443ac-a7c3-48b5-a22a-f38692bccdb6" containerName="main" Jul 08 18:18:12.685662 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.685595 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="513443ac-a7c3-48b5-a22a-f38692bccdb6" containerName="main" Jul 08 18:18:12.685662 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.685609 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="10fca1e5-6520-45c4-a57f-9bb6d692c497" containerName="main" Jul 08 18:18:12.688810 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.688794 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.691459 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.691433 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gdpqb\"/\"kube-root-ca.crt\"" Jul 08 18:18:12.692379 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.692360 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gdpqb\"/\"openshift-service-ca.crt\"" Jul 08 18:18:12.692509 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.692396 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gdpqb\"/\"default-dockercfg-tn67x\"" Jul 08 18:18:12.698422 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.698380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jqbr\" (UniqueName: \"kubernetes.io/projected/b5b0079d-5565-4f8f-803a-b33d54b6429b-kube-api-access-4jqbr\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.698535 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.698457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-podres\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.698535 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.698513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-proc\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.698651 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.698560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-lib-modules\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.698651 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.698612 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-sys\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.699363 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.699345 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss"] Jul 08 18:18:12.800177 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jqbr\" (UniqueName: \"kubernetes.io/projected/b5b0079d-5565-4f8f-803a-b33d54b6429b-kube-api-access-4jqbr\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.800393 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800196 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-podres\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.800393 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-proc\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.800393 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-lib-modules\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.800393 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800282 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-sys\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.800393 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800366 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-podres\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.800393 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800387 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-lib-modules\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.800393 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800376 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-sys\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.800751 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.800440 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5b0079d-5565-4f8f-803a-b33d54b6429b-proc\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:12.814787 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:12.814761 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jqbr\" (UniqueName: \"kubernetes.io/projected/b5b0079d-5565-4f8f-803a-b33d54b6429b-kube-api-access-4jqbr\") pod \"perf-node-gather-daemonset-5x7ss\" (UID: \"b5b0079d-5565-4f8f-803a-b33d54b6429b\") " pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:13.000489 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:13.000400 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:13.138286 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:13.138264 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss"] Jul 08 18:18:13.140302 ip-10-0-129-217 kubenswrapper[2580]: W0708 18:18:13.140275 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb5b0079d_5565_4f8f_803a_b33d54b6429b.slice/crio-ae4b3c59b1099725873ae12e77f4ee6ecd4373fca126250d9ee95eadfbe23e32 WatchSource:0}: Error finding container ae4b3c59b1099725873ae12e77f4ee6ecd4373fca126250d9ee95eadfbe23e32: Status 404 returned error can't find the container with id ae4b3c59b1099725873ae12e77f4ee6ecd4373fca126250d9ee95eadfbe23e32 Jul 08 18:18:13.237594 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:13.237526 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" event={"ID":"b5b0079d-5565-4f8f-803a-b33d54b6429b","Type":"ContainerStarted","Data":"29e8fcb651125b57c9f84516a7ecd121942bb3af16f04cde232058e28597dc61"} Jul 08 18:18:13.237594 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:13.237600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" event={"ID":"b5b0079d-5565-4f8f-803a-b33d54b6429b","Type":"ContainerStarted","Data":"ae4b3c59b1099725873ae12e77f4ee6ecd4373fca126250d9ee95eadfbe23e32"} Jul 08 18:18:13.237819 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:13.237713 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:13.260821 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:13.260719 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" podStartSLOduration=1.260703523 podStartE2EDuration="1.260703523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:13.259795821 +0000 UTC m=+2342.323318652" watchObservedRunningTime="2026-07-08 18:18:13.260703523 +0000 UTC m=+2342.324226352" Jul 08 18:18:15.102404 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:15.102370 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-575d77984-d2pw9_0b3e1978-4c68-46c0-92ff-11bb8972c638/console/0.log" Jul 08 18:18:17.148822 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:17.148790 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dgvdn_dffdb6f2-534a-4ffc-be18-f5c7a5246a45/dns/0.log" Jul 08 18:18:17.212418 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:17.212392 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dgvdn_dffdb6f2-534a-4ffc-be18-f5c7a5246a45/kube-rbac-proxy/0.log" Jul 08 18:18:17.401026 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:17.400947 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-44b9m_c255717f-5f5f-49fd-b0b7-8b320723e799/dns-node-resolver/0.log" Jul 08 18:18:18.179095 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:18.179061 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lb6x5_51eeea3f-4533-4d4f-9048-20d7262dc02d/node-ca/0.log" Jul 08 18:18:19.252855 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:19.252830 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gdpqb/perf-node-gather-daemonset-5x7ss" Jul 08 18:18:19.564371 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:19.564344 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64959b45fc-8nwtd_f8a9e5ca-5edf-4afe-9ddb-b02b5f406152/kube-auth-proxy/0.log" Jul 08 18:18:19.729770 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:19.729734 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-72svh_3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6/payload-pre-processing/0.log" Jul 08 18:18:19.804364 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:19.804335 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-4trf4_4633c308-1619-439d-a657-9a0a835309d3/payload-processing/0.log" Jul 08 18:18:20.602825 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:20.602792 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-89lm7_17943a25-4001-43e7-a778-fa5103f90de2/serve-healthcheck-canary/0.log" Jul 08 18:18:21.410738 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:21.410707 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6pkt6_f32cc2e4-7870-4b80-8087-b9d43a8dda10/kube-rbac-proxy/0.log" Jul 08 18:18:21.442303 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:21.442278 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6pkt6_f32cc2e4-7870-4b80-8087-b9d43a8dda10/exporter/0.log" Jul 08 18:18:21.476378 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:21.476351 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6pkt6_f32cc2e4-7870-4b80-8087-b9d43a8dda10/extractor/0.log" Jul 08 18:18:23.413684 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:23.413659 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64959b45fc-8nwtd_f8a9e5ca-5edf-4afe-9ddb-b02b5f406152/kube-auth-proxy/0.log" Jul 08 18:18:23.541042 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:23.541019 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-72svh_3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6/payload-pre-processing/0.log" Jul 08 18:18:23.603497 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:23.603449 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-4trf4_4633c308-1619-439d-a657-9a0a835309d3/payload-processing/0.log" Jul 08 18:18:24.739123 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:24.739088 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b69778487-j2kgn_d465147e-20d0-45aa-8702-2ed592770dbe/manager/0.log" Jul 08 18:18:25.074951 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:25.074918 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64959b45fc-8nwtd_f8a9e5ca-5edf-4afe-9ddb-b02b5f406152/kube-auth-proxy/0.log" Jul 08 18:18:25.078166 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:25.078134 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dcf78d495-69vcq_24cb335e-a12b-46ba-8f55-6abd43401e90/manager/0.log" Jul 08 18:18:25.108620 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:25.108594 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dcf78d495-69vcq_24cb335e-a12b-46ba-8f55-6abd43401e90/copy-manifests/0.log" Jul 08 18:18:25.202777 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:25.202746 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-72svh_3956ee0a-e1e8-42f9-a2d7-cb64ff9b63a6/payload-pre-processing/0.log" Jul 08 18:18:25.269016 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:25.268990 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-4trf4_4633c308-1619-439d-a657-9a0a835309d3/payload-processing/0.log" Jul 08 18:18:26.499431 ip-10-0-129-217 kubenswrapper[2580]: I0708 18:18:26.499402 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7fbcbf7958-ddhkz_327f3329-448c-4183-9a12-1ae38cca041a/manager/0.log"