Jul 01 22:29:58.342211 ip-10-0-141-190 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 01 22:29:58.342226 ip-10-0-141-190 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 01 22:29:58.342235 ip-10-0-141-190 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 01 22:29:58.342713 ip-10-0-141-190 systemd[1]: Failed to start Kubernetes Kubelet. Jul 01 22:30:08.423837 ip-10-0-141-190 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 01 22:30:08.423857 ip-10-0-141-190 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot bc2507d9597f4142848e16baba0014a0 -- Jul 01 22:32:08.986384 ip-10-0-141-190 systemd[1]: Starting Kubernetes Kubelet... Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.410817 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417874 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417888 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417893 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417900 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417905 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417909 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417914 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:32:09.453502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417919 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417922 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417926 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417930 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417934 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417939 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417945 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417949 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417953 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417958 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417962 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417965 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417970 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417974 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417978 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417985 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417993 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.417998 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418003 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:32:09.454814 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418007 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418011 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418016 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418020 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418024 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418029 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418034 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418038 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418042 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418059 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418064 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418069 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418074 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418079 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418083 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418087 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418091 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418095 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418099 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418103 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:32:09.455719 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418107 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418113 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418117 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418120 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418125 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418129 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418133 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418139 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418144 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418148 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418153 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418157 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418161 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418166 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418171 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418175 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418180 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418184 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418188 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418193 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:32:09.456614 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418197 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418202 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418207 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418211 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418217 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418227 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418234 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418238 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418243 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418247 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418252 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418256 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418261 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418265 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418268 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418272 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418277 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418280 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418285 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418288 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:32:09.457512 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418293 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418297 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418302 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418306 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418310 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418314 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418318 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418322 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418326 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418331 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418336 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418340 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.418344 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419332 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419342 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419347 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419352 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419356 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419361 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419365 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:32:09.458390 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419369 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419373 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419377 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419381 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419385 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419389 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419393 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419397 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419401 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419405 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419409 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419412 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419418 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419422 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419427 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419431 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419436 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419440 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419444 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:32:09.462459 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419451 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419456 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419461 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419469 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419473 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419478 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419482 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419486 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419491 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419495 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419500 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419505 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419509 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419514 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419518 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419523 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419527 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419531 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419537 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419541 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:32:09.463365 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419545 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419548 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419552 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419556 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419560 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419563 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419568 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419571 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419576 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419579 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419583 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419587 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419591 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419595 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419599 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419605 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419608 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419612 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419616 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419621 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:32:09.470465 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419624 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419628 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419632 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419636 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419640 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419644 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419648 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419654 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419659 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419664 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419672 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419676 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419681 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419687 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419692 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419696 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419700 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419704 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419708 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:32:09.471546 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419712 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419716 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419720 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419724 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419728 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419732 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419736 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419740 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419745 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419749 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419753 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419757 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419761 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.419764 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420351 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420359 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420363 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420367 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420373 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:32:09.472432 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420378 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420382 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420386 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420391 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420398 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420402 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420406 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420410 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420414 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420418 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420422 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420426 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420430 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420435 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420438 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420443 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420447 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420451 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420454 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420461 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:32:09.473328 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420466 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420473 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420477 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420482 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420486 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420490 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420494 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420497 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420502 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420506 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420510 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420514 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420519 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420523 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420527 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420531 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420537 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420541 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420545 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420549 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:32:09.474219 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420555 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420559 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420564 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420569 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420573 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420577 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420580 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420584 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420589 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420592 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420596 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420600 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420604 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420609 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420613 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420617 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420621 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420625 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420629 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420632 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:32:09.475307 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420636 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420640 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420644 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420649 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420653 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420657 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420661 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420665 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420670 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420674 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420677 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420681 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420686 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420689 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420693 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420697 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420701 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420704 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420708 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420713 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:32:09.476392 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420716 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420721 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420725 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420729 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420733 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420739 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420745 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420750 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420755 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420759 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420763 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420766 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420770 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.420775 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420863 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420882 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420890 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420897 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420904 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420909 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420917 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 22:32:09.477265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420923 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420928 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420932 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420938 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420943 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420948 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420953 2581 flags.go:64] FLAG: --cgroup-root="" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420958 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420963 2581 flags.go:64] FLAG: --client-ca-file="" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420967 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420972 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420979 2581 flags.go:64] FLAG: --cluster-domain="" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420983 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420988 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420994 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.420999 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421007 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421011 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421016 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421021 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421026 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421031 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421036 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421040 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 22:32:09.478143 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421045 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421066 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421072 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421076 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421080 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421085 2581 flags.go:64] FLAG: --enable-server="true" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421091 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421100 2581 flags.go:64] FLAG: --event-burst="100" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421105 2581 flags.go:64] FLAG: --event-qps="50" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421110 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421115 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421119 2581 flags.go:64] FLAG: --eviction-hard="" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421125 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421130 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421135 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421139 2581 flags.go:64] FLAG: --eviction-soft="" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421144 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421148 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421152 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421157 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421162 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421166 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421171 2581 flags.go:64] FLAG: --feature-gates="" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421177 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421183 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 22:32:09.479261 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421187 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421192 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421197 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421201 2581 flags.go:64] FLAG: --help="false" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421206 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421210 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421215 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421220 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421225 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421231 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421236 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421240 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421245 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421249 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421256 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421261 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421266 2581 flags.go:64] FLAG: --kube-reserved="" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421271 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421275 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421280 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421284 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421289 2581 flags.go:64] FLAG: --lock-file="" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421293 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421298 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 22:32:09.480401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421302 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421311 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421315 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421319 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421324 2581 flags.go:64] FLAG: --logging-format="text" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421329 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421334 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421340 2581 flags.go:64] FLAG: --manifest-url="" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421344 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421352 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421357 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421363 2581 flags.go:64] FLAG: --max-pods="110" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421368 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421372 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421377 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421382 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421386 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421391 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421396 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421406 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421411 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421416 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421423 2581 flags.go:64] FLAG: --pod-cidr="" Jul 01 22:32:09.568434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421428 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 22:32:09.495583 ip-10-0-141-190 systemd[1]: Started Kubernetes Kubelet. Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421437 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421441 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421447 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421451 2581 flags.go:64] FLAG: --port="10250" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421456 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421461 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0809f2f3f36b44c77" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421466 2581 flags.go:64] FLAG: --qos-reserved="" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421471 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421475 2581 flags.go:64] FLAG: --register-node="true" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421480 2581 flags.go:64] FLAG: --register-with-taints="" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421486 2581 flags.go:64] FLAG: --registry-burst="10" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421490 2581 flags.go:64] FLAG: --registry-qps="5" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421495 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421499 2581 flags.go:64] FLAG: --reserved-memory="" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421506 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421512 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421517 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421522 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421526 2581 flags.go:64] FLAG: --runonce="false" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421531 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421535 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421540 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421544 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421549 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421554 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 22:32:09.569990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421558 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421563 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421568 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421572 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421577 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421582 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421589 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421594 2581 flags.go:64] FLAG: --system-cgroups="" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421599 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421607 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421612 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421616 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421623 2581 flags.go:64] FLAG: --tls-min-version="" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421628 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421633 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421637 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421641 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421646 2581 flags.go:64] FLAG: --v="2" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421652 2581 flags.go:64] FLAG: --version="false" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421658 2581 flags.go:64] FLAG: --vmodule="" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421664 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.421670 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421829 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421834 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:32:09.571904 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421840 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421845 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421851 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421855 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421860 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421864 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421868 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421872 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421878 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421883 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421887 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421891 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421895 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421899 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421905 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421909 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421914 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421918 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421922 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:32:09.573471 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421926 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421930 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421934 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421938 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421942 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421947 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421951 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421955 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421959 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421963 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421966 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421972 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421976 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421980 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421984 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421988 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421992 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.421996 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422000 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422003 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:32:09.574685 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422007 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422011 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422015 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422019 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422023 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422027 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422031 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422037 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422040 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422044 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422061 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422066 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422070 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422074 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422078 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422082 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422086 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422090 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422094 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422098 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:32:09.575867 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422102 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422106 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422110 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422116 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422120 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422124 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422128 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422132 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422137 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422142 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422148 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422152 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422156 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422161 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422165 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422169 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422173 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422177 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422181 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:32:09.577071 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422186 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422190 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422194 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422198 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422203 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422207 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422211 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422215 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422219 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422223 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422227 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422231 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422235 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422239 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422244 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422248 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422254 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422259 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:32:09.578030 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.422264 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.422271 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.431211 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.431227 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.431244 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.431251 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431359 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431364 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431368 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431371 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431374 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431377 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431381 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431384 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431387 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:32:09.578988 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431389 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431395 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431399 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431402 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431404 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431407 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431410 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431413 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431416 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431418 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431421 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431424 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431427 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431429 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431436 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431441 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431444 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431447 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431450 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:32:09.582716 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431453 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431457 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431460 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431463 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431466 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431469 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431472 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431477 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431480 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431483 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431485 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431488 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431491 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431495 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431498 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431501 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431504 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431507 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431510 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:32:09.583927 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431513 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431519 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431522 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431524 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431527 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431530 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431533 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431536 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431539 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431542 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431545 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431548 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431552 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431559 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431567 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431570 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431573 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431576 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431579 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431582 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:32:09.591654 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431585 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431588 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431591 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431593 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431596 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431602 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431609 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431612 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431615 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431633 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431636 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431639 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431642 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431740 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431744 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431747 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431750 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431752 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431755 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431758 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:32:09.592832 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431760 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431763 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431766 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431769 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431771 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431774 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431777 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431780 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431787 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431790 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431793 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431797 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.431803 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431933 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431938 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:32:09.593728 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431941 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431944 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431947 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431951 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431954 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431956 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431959 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431961 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431964 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431966 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431969 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431973 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431977 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431980 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431982 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431985 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431988 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431990 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431993 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:32:09.594686 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431995 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.431998 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432001 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432004 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432006 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432009 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432012 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432015 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432017 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432020 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432024 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432027 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432030 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432032 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432035 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432037 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432041 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432044 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432063 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432067 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432069 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:32:09.595649 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432072 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432075 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432077 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432080 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432083 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432086 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432088 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432091 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432093 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432096 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432098 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432101 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432103 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432114 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432117 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432119 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432122 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432124 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432127 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:32:09.596828 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432130 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432132 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432136 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432140 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432142 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432145 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432147 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432150 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432154 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432158 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432161 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432164 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432168 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432171 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432174 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432177 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432180 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432184 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432186 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:32:09.597787 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432189 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432192 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432194 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432197 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432199 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432202 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432204 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432207 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432209 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432212 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432215 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432217 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432220 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432222 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432225 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432229 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432231 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432234 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:32:09.598772 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:09.432236 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.432241 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.432343 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.436705 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.438288 2581 server.go:1019] "Starting client certificate rotation" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.438386 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.438415 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.441015 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.445038 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.455071 2581 log.go:25] "Validated CRI v1 runtime API" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.459454 2581 log.go:25] "Validated CRI v1 image API" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.460176 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 22:32:09.599751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.465136 2581 fs.go:135] Filesystem UUIDs: map[1b235eae-760d-4200-9df1-289f40527a0d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d5a8cce5-959b-4441-b697-32b680f85d42:/dev/nvme0n1p3] Jul 01 22:32:09.600454 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.465151 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 22:32:09.600693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.469928 2581 manager.go:216] Machine: {Timestamp:2026-07-01 22:32:09.46852604 +0000 UTC m=+0.355783989 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100427 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ddb15df9cf60aacc4f59d11fe5df8 SystemUUID:ec2ddb15-df9c-f60a-acc4-f59d11fe5df8 BootID:bc2507d9-597f-4142-848e-16baba0014a0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:06:ac:b8:18:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:06:ac:b8:18:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:f0:c1:fa:ee:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 22:32:09.600693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.470024 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 22:32:09.600693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.470146 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 22:32:09.600693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.472209 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 22:32:09.600693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.472236 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-190.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 22:32:09.600693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.473621 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 22:32:09.600693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.473630 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 22:32:09.600693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.473637 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.474344 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.475245 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.478655 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.478747 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.479549 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.479566 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.479578 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.479586 2581 kubelet.go:398] "Adding apiserver pod source" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.479594 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.479863 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.479881 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.483274 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.483292 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.489765 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.489793 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490420 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490432 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490438 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490444 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490452 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490458 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490473 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490478 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490484 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490489 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490497 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.490505 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.491372 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.491383 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.493848 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.493851 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-190.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.493870 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-190.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.494696 2581 server.go:1268] "Started kubelet" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.494769 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.494795 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.494867 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.495923 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 22:32:09.601379 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.497355 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.503745 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-190.ec2.internal.18be4b26198aeef4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-190.ec2.internal,UID:ip-10-0-141-190.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-190.ec2.internal,},FirstTimestamp:2026-07-01 22:32:09.494671092 +0000 UTC m=+0.381929039,LastTimestamp:2026-07-01 22:32:09.494671092 +0000 UTC m=+0.381929039,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-190.ec2.internal,}" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.504863 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.504880 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.504927 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.505951 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.506001 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.506015 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.506132 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.506140 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.507328 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tgw5f" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.507583 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.507604 2581 factory.go:55] Registering systemd factory Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.507613 2581 factory.go:223] Registration of the systemd container factory successfully Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.507922 2581 factory.go:153] Registering CRI-O factory Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.507936 2581 factory.go:223] Registration of the crio container factory successfully Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.507995 2581 factory.go:103] Registering Raw factory Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.508038 2581 manager.go:1196] Started watching for new ooms in manager Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.509305 2581 manager.go:349] Starting recovery of all containers Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.511528 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-190.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.511597 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.516998 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tgw5f" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.523255 2581 manager.go:354] Recovery completed Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.525169 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.530616 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.537195 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.537229 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.537241 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.538131 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.538146 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.538164 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.538693 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.539321 2581 policy_none.go:49] "None policy: Start" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.539338 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 22:32:09.603070 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.539353 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 22:32:09.604739 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.539481 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:32:09.604739 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.539507 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:32:09.604739 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.539522 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:32:09.604739 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.540523 2581 policy_none.go:47] "Start" Jul 01 22:32:09.604739 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.541587 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-190.ec2.internal.18be4b261c140935 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-190.ec2.internal,UID:ip-10-0-141-190.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-190.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-190.ec2.internal,},FirstTimestamp:2026-07-01 22:32:09.537210677 +0000 UTC m=+0.424468631,LastTimestamp:2026-07-01 22:32:09.537210677 +0000 UTC m=+0.424468631,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-190.ec2.internal,}" Jul 01 22:32:09.605688 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.605229 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.613852 2581 manager.go:337] "Starting Device Plugin manager" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.613888 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.613902 2581 server.go:87] "Starting device plugin registration server" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.614228 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.614246 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.614427 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.614518 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.614529 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.615062 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.615137 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.658476 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.658513 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.658538 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.658572 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.661382 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.714859 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.716458 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.716488 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.716501 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.716526 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.720890 2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-190.ec2.internal\" not found" node="ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.728096 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.728113 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-190.ec2.internal\": node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.758433 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.759499 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal"] Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.759558 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.760404 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.760431 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.760447 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762018 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762157 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762190 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762792 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762794 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762838 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:32:09.812081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762850 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762817 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.762887 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.764286 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.764311 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.764990 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.765014 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.765024 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.801950 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-190.ec2.internal\" not found" node="ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.806763 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-190.ec2.internal\" not found" node="ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.807825 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d6d3eb16eb91756b002dffb0f47321a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal\" (UID: \"1d6d3eb16eb91756b002dffb0f47321a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.807847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d6d3eb16eb91756b002dffb0f47321a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal\" (UID: \"1d6d3eb16eb91756b002dffb0f47321a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:09.813389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.807866 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a76c7746494e50a2be891e8cc385fe8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-190.ec2.internal\" (UID: \"1a76c7746494e50a2be891e8cc385fe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.022446 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.858585 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:10.022446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.908533 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d6d3eb16eb91756b002dffb0f47321a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal\" (UID: \"1d6d3eb16eb91756b002dffb0f47321a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.022446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.908563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d6d3eb16eb91756b002dffb0f47321a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal\" (UID: \"1d6d3eb16eb91756b002dffb0f47321a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.022446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.908581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a76c7746494e50a2be891e8cc385fe8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-190.ec2.internal\" (UID: \"1a76c7746494e50a2be891e8cc385fe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.022446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.908627 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a76c7746494e50a2be891e8cc385fe8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-190.ec2.internal\" (UID: \"1a76c7746494e50a2be891e8cc385fe8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.022446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.908642 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d6d3eb16eb91756b002dffb0f47321a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal\" (UID: \"1d6d3eb16eb91756b002dffb0f47321a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.022446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:09.908698 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d6d3eb16eb91756b002dffb0f47321a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal\" (UID: \"1d6d3eb16eb91756b002dffb0f47321a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.022446 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:09.959598 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:10.184508 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.063352 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:10.184508 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.106575 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.184508 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.112044 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.184508 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.164037 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:10.399884 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.265938 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:10.399884 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.356258 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:32:10.399884 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.366107 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-190.ec2.internal\" not found" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.412693 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.438420 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.438538 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.438538 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.438538 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.482298 2581 apiserver.go:52] "Watching apiserver" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.505133 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.505143 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.505267 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.505479 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-2dxrj","openshift-network-diagnostics/network-check-target-gwf6l","openshift-ovn-kubernetes/ovnkube-node-95fqr","kube-system/konnectivity-agent-lrs62","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz","openshift-cluster-node-tuning-operator/tuned-m7krv","openshift-dns/node-resolver-c92kx","openshift-image-registry/node-ca-2pxjt","openshift-multus/multus-additional-cni-plugins-fhm8h","openshift-multus/multus-pswsl","openshift-network-operator/iptables-alerter-zxcgk","kube-system/global-pull-secret-syncer-4g9k9"] Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.506923 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.506990 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.508095 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.508145 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.512569 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.512611 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmq8v\" (UniqueName: \"kubernetes.io/projected/a25f676f-13ae-48b5-b613-04af7f4f078e-kube-api-access-dmq8v\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.514159 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.516843 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.518117 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.519357 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.520177 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 22:32:10.583015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.520509 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5864j\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.520524 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.520594 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.521814 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.522215 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.522251 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.522265 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.522474 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 22:27:10 +0000 UTC" deadline="2028-04-05 16:56:24.14824147 +0000 UTC" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.522492 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15450h24m13.625751825s" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.523135 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.524550 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.526054 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.526189 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.526230 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8r5sg\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.526734 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.527789 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.527859 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528206 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528206 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cf9bw\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528268 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528225 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528450 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528506 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528573 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528610 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528664 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528676 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528705 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.528759 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.529206 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w28cv\"" Jul 01 22:32:10.584495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.529211 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.529286 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.529320 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.529438 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.529529 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jmhc9\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.529833 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.529935 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.530228 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.530360 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mws2m\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.530578 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8hlhw\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.530651 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.530714 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jtpp9\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.535344 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal"] Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.535787 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.535847 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.535845 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.536775 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-flmzs\"" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.547289 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.572624 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal"] Jul 01 22:32:10.585654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.572660 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.607463 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612820 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09e5d35c-7bb6-4eda-b264-e25731987f58-host\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c53760f-41eb-4097-8252-7002953475fc-cni-binary-copy\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-multus-certs\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-node-log\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovnkube-script-lib\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-socket-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612932 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-sys-fs\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612947 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612964 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-os-release\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-socket-dir-parent\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.612993 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-k8s-cni-cncf-io\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-kubelet\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.705826 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhfrw\" (UniqueName: \"kubernetes.io/projected/31506d3f-0743-4684-8b09-aa952f816530-kube-api-access-lhfrw\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613058 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613073 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-lib-modules\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcqqc\" (UniqueName: \"kubernetes.io/projected/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-kube-api-access-lcqqc\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613118 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-cni-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613139 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-cni-bin\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613160 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovnkube-config\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613176 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09e5d35c-7bb6-4eda-b264-e25731987f58-serviceca\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613191 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-var-lib-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613207 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-etc-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613222 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613238 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-netns\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613252 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-conf-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.707425 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-os-release\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613282 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-systemd\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-tuned\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613319 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba4ff25c-4df9-47ff-8c80-5e1a10867213-hosts-file\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-iptables-alerter-script\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-run-netns\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51564124-a668-464c-80ef-2985c3c6c109-agent-certs\") pod \"konnectivity-agent-lrs62\" (UID: \"51564124-a668-464c-80ef-2985c3c6c109\") " pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613386 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-cnibin\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613404 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-kubelet\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613422 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m64r5\" (UniqueName: \"kubernetes.io/projected/9c53760f-41eb-4097-8252-7002953475fc-kube-api-access-m64r5\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613438 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-systemd-units\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613467 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovn-node-metrics-cert\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613483 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88f7x\" (UniqueName: \"kubernetes.io/projected/8f450d1f-663d-4db7-89f0-b2622c0eff7e-kube-api-access-88f7x\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.708082 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613498 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-sys\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613516 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkmp5\" (UniqueName: \"kubernetes.io/projected/ba4ff25c-4df9-47ff-8c80-5e1a10867213-kube-api-access-rkmp5\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613532 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-system-cni-dir\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613548 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-cnibin\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613562 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613578 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysconfig\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613593 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-kubernetes\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613609 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-run\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613623 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-host\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613638 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-host-slash\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-ovn\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/889ac5e9-344c-4000-8257-66a0ef85e0eb-kubelet-config\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613692 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-modprobe-d\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613709 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysctl-conf\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.708819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fskz7\" (UniqueName: \"kubernetes.io/projected/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-kube-api-access-fskz7\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-env-overrides\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613756 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba4ff25c-4df9-47ff-8c80-5e1a10867213-tmp-dir\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613773 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmq8v\" (UniqueName: \"kubernetes.io/projected/a25f676f-13ae-48b5-b613-04af7f4f078e-kube-api-access-dmq8v\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613809 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-tmp\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613824 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-cni-bin\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-cni-multus\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-cni-netd\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-device-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5v6w\" (UniqueName: \"kubernetes.io/projected/88b2f416-649a-4170-bd37-210de6580d47-kube-api-access-g5v6w\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613909 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613924 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/889ac5e9-344c-4000-8257-66a0ef85e0eb-dbus\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.709435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613940 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c53760f-41eb-4097-8252-7002953475fc-multus-daemon-config\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613960 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613976 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-etc-selinux\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.613992 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysctl-d\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67zfv\" (UniqueName: \"kubernetes.io/projected/09e5d35c-7bb6-4eda-b264-e25731987f58-kube-api-access-67zfv\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-hostroot\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614060 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-etc-kubernetes\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614078 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-systemd\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51564124-a668-464c-80ef-2985c3c6c109-konnectivity-ca\") pod \"konnectivity-agent-lrs62\" (UID: \"51564124-a668-464c-80ef-2985c3c6c109\") " pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614113 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-var-lib-kubelet\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614252 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-system-cni-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614271 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-slash\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614288 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-log-socket\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.710246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.710935 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.614318 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-registration-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.710935 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.614488 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:10.710935 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.614541 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs podName:a25f676f-13ae-48b5-b613-04af7f4f078e nodeName:}" failed. No retries permitted until 2026-07-01 22:32:11.114519092 +0000 UTC m=+2.001777029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs") pod "network-metrics-daemon-2dxrj" (UID: "a25f676f-13ae-48b5-b613-04af7f4f078e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:10.710935 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.633616 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 22:32:10.710935 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.636825 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmq8v\" (UniqueName: \"kubernetes.io/projected/a25f676f-13ae-48b5-b613-04af7f4f078e-kube-api-access-dmq8v\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:10.710935 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.647003 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8s8sx" Jul 01 22:32:10.710935 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.662179 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8s8sx" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysctl-d\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67zfv\" (UniqueName: \"kubernetes.io/projected/09e5d35c-7bb6-4eda-b264-e25731987f58-kube-api-access-67zfv\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715324 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-hostroot\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715347 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-etc-kubernetes\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715366 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-systemd\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715384 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51564124-a668-464c-80ef-2985c3c6c109-konnectivity-ca\") pod \"konnectivity-agent-lrs62\" (UID: \"51564124-a668-464c-80ef-2985c3c6c109\") " pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-var-lib-kubelet\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715414 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-hostroot\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715432 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-systemd\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715427 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-system-cni-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715423 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-etc-kubernetes\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715475 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysctl-d\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715494 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-var-lib-kubelet\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715523 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-slash\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.828342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715537 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-system-cni-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-log-socket\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-slash\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715578 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-log-socket\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715602 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715628 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-registration-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715645 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09e5d35c-7bb6-4eda-b264-e25731987f58-host\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c53760f-41eb-4097-8252-7002953475fc-cni-binary-copy\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715701 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715709 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-registration-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715712 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09e5d35c-7bb6-4eda-b264-e25731987f58-host\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715749 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-multus-certs\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715759 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-multus-certs\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715784 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-node-log\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovnkube-script-lib\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.829495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715841 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-node-log\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715839 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-socket-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-sys-fs\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-os-release\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715944 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-socket-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715951 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-socket-dir-parent\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715970 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-k8s-cni-cncf-io\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.715999 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51564124-a668-464c-80ef-2985c3c6c109-konnectivity-ca\") pod \"konnectivity-agent-lrs62\" (UID: \"51564124-a668-464c-80ef-2985c3c6c109\") " pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716016 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-k8s-cni-cncf-io\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716005 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-sys-fs\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716089 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-os-release\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-kubelet\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716128 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhfrw\" (UniqueName: \"kubernetes.io/projected/31506d3f-0743-4684-8b09-aa952f816530-kube-api-access-lhfrw\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.830396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716104 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-socket-dir-parent\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716172 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716192 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-kubelet\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-lib-modules\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716256 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcqqc\" (UniqueName: \"kubernetes.io/projected/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-kube-api-access-lcqqc\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716281 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-cni-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-cni-bin\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716324 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c53760f-41eb-4097-8252-7002953475fc-cni-binary-copy\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716328 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovnkube-config\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716470 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-cni-bin\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09e5d35c-7bb6-4eda-b264-e25731987f58-serviceca\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716525 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-cni-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716527 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-var-lib-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716558 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-var-lib-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716526 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.831152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-etc-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-netns\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716613 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-etc-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.716684 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716730 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-run-netns\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.716751 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret podName:889ac5e9-344c-4000-8257-66a0ef85e0eb nodeName:}" failed. No retries permitted until 2026-07-01 22:32:11.216732441 +0000 UTC m=+2.103990377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret") pod "global-pull-secret-syncer-4g9k9" (UID: "889ac5e9-344c-4000-8257-66a0ef85e0eb") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716812 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-conf-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716845 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovnkube-config\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.717118 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09e5d35c-7bb6-4eda-b264-e25731987f58-serviceca\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.718794 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovnkube-script-lib\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.716661 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-multus-conf-dir\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721499 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-lib-modules\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-os-release\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.831753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721541 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-systemd\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721547 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-os-release\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-tuned\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba4ff25c-4df9-47ff-8c80-5e1a10867213-hosts-file\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721594 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-systemd\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-iptables-alerter-script\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-run-netns\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721648 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51564124-a668-464c-80ef-2985c3c6c109-agent-certs\") pod \"konnectivity-agent-lrs62\" (UID: \"51564124-a668-464c-80ef-2985c3c6c109\") " pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721676 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-cnibin\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721699 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-run-netns\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721705 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba4ff25c-4df9-47ff-8c80-5e1a10867213-hosts-file\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721703 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-kubelet\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721759 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m64r5\" (UniqueName: \"kubernetes.io/projected/9c53760f-41eb-4097-8252-7002953475fc-kube-api-access-m64r5\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721770 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-cnibin\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721775 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-kubelet\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.832356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-systemd-units\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721923 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721951 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovn-node-metrics-cert\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721976 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88f7x\" (UniqueName: \"kubernetes.io/projected/8f450d1f-663d-4db7-89f0-b2622c0eff7e-kube-api-access-88f7x\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722001 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-sys\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.721978 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-systemd-units\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722047 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-iptables-alerter-script\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkmp5\" (UniqueName: \"kubernetes.io/projected/ba4ff25c-4df9-47ff-8c80-5e1a10867213-kube-api-access-rkmp5\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722000 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-openvswitch\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722123 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-sys\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722164 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-system-cni-dir\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-cnibin\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722205 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-system-cni-dir\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.832923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722261 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-cnibin\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722293 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysconfig\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-kubernetes\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722348 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysconfig\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722366 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-run\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722367 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31506d3f-0743-4684-8b09-aa952f816530-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722390 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-host\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722392 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-kubernetes\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722416 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-host-slash\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722419 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-run\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722459 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-host\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-ovn\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722480 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-host-slash\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722494 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/889ac5e9-344c-4000-8257-66a0ef85e0eb-kubelet-config\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722501 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-run-ovn\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.833515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722520 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-modprobe-d\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysctl-conf\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722551 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/889ac5e9-344c-4000-8257-66a0ef85e0eb-kubelet-config\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722578 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fskz7\" (UniqueName: \"kubernetes.io/projected/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-kube-api-access-fskz7\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-env-overrides\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722621 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-modprobe-d\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722631 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba4ff25c-4df9-47ff-8c80-5e1a10867213-tmp-dir\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722640 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-sysctl-conf\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722659 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722706 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-tmp\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-cni-bin\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722790 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-cni-multus\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722838 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-cni-netd\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.834081 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722864 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-device-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722892 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5v6w\" (UniqueName: \"kubernetes.io/projected/88b2f416-649a-4170-bd37-210de6580d47-kube-api-access-g5v6w\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722946 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/889ac5e9-344c-4000-8257-66a0ef85e0eb-dbus\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722953 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba4ff25c-4df9-47ff-8c80-5e1a10867213-tmp-dir\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.722983 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c53760f-41eb-4097-8252-7002953475fc-multus-daemon-config\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723007 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-device-dir\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-etc-selinux\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723055 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8f450d1f-663d-4db7-89f0-b2622c0eff7e-env-overrides\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723068 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/889ac5e9-344c-4000-8257-66a0ef85e0eb-dbus\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723098 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723022 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-cni-netd\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723309 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f450d1f-663d-4db7-89f0-b2622c0eff7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.834629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723402 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88b2f416-649a-4170-bd37-210de6580d47-etc-selinux\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723481 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-cni-bin\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723526 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9c53760f-41eb-4097-8252-7002953475fc-host-var-lib-cni-multus\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723535 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.723604 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9c53760f-41eb-4097-8252-7002953475fc-multus-daemon-config\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.724257 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/31506d3f-0743-4684-8b09-aa952f816530-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.724518 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51564124-a668-464c-80ef-2985c3c6c109-agent-certs\") pod \"konnectivity-agent-lrs62\" (UID: \"51564124-a668-464c-80ef-2985c3c6c109\") " pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.724643 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-etc-tuned\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.724690 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8f450d1f-663d-4db7-89f0-b2622c0eff7e-ovn-node-metrics-cert\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.725083 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-tmp\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.728291 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.728317 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.728333 2581 projected.go:196] Error preparing data for projected volume kube-api-access-c4df5 for pod openshift-network-diagnostics/network-check-target-gwf6l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:10.728416 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5 podName:fd36e128-b67d-4e16-acdf-b755b5fdbbca nodeName:}" failed. No retries permitted until 2026-07-01 22:32:11.22839553 +0000 UTC m=+2.115653480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c4df5" (UniqueName: "kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5") pod "network-check-target-gwf6l" (UID: "fd36e128-b67d-4e16-acdf-b755b5fdbbca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:10.835115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.730249 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67zfv\" (UniqueName: \"kubernetes.io/projected/09e5d35c-7bb6-4eda-b264-e25731987f58-kube-api-access-67zfv\") pod \"node-ca-2pxjt\" (UID: \"09e5d35c-7bb6-4eda-b264-e25731987f58\") " pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.734411 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcqqc\" (UniqueName: \"kubernetes.io/projected/1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5-kube-api-access-lcqqc\") pod \"tuned-m7krv\" (UID: \"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5\") " pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.735086 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhfrw\" (UniqueName: \"kubernetes.io/projected/31506d3f-0743-4684-8b09-aa952f816530-kube-api-access-lhfrw\") pod \"multus-additional-cni-plugins-fhm8h\" (UID: \"31506d3f-0743-4684-8b09-aa952f816530\") " pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.738744 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88f7x\" (UniqueName: \"kubernetes.io/projected/8f450d1f-663d-4db7-89f0-b2622c0eff7e-kube-api-access-88f7x\") pod \"ovnkube-node-95fqr\" (UID: \"8f450d1f-663d-4db7-89f0-b2622c0eff7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.740632 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5v6w\" (UniqueName: \"kubernetes.io/projected/88b2f416-649a-4170-bd37-210de6580d47-kube-api-access-g5v6w\") pod \"aws-ebs-csi-driver-node-dzdtz\" (UID: \"88b2f416-649a-4170-bd37-210de6580d47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.742478 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m64r5\" (UniqueName: \"kubernetes.io/projected/9c53760f-41eb-4097-8252-7002953475fc-kube-api-access-m64r5\") pod \"multus-pswsl\" (UID: \"9c53760f-41eb-4097-8252-7002953475fc\") " pod="openshift-multus/multus-pswsl" Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.744482 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkmp5\" (UniqueName: \"kubernetes.io/projected/ba4ff25c-4df9-47ff-8c80-5e1a10867213-kube-api-access-rkmp5\") pod \"node-resolver-c92kx\" (UID: \"ba4ff25c-4df9-47ff-8c80-5e1a10867213\") " pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.744561 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fskz7\" (UniqueName: \"kubernetes.io/projected/b32433b1-92f3-4ac7-ab9a-0ad2c373c576-kube-api-access-fskz7\") pod \"iptables-alerter-zxcgk\" (UID: \"b32433b1-92f3-4ac7-ab9a-0ad2c373c576\") " pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:10.749150 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6d3eb16eb91756b002dffb0f47321a.slice/crio-fe4411a13d819d045d063bc7cb31325182e88fe74a670df855880308e1803d5e WatchSource:0}: Error finding container fe4411a13d819d045d063bc7cb31325182e88fe74a670df855880308e1803d5e: Status 404 returned error can't find the container with id fe4411a13d819d045d063bc7cb31325182e88fe74a670df855880308e1803d5e Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:10.749596 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a76c7746494e50a2be891e8cc385fe8.slice/crio-94540cdda281434c4febf061b5f70816b5ee824f5f3f34280ee1f646796d571c WatchSource:0}: Error finding container 94540cdda281434c4febf061b5f70816b5ee824f5f3f34280ee1f646796d571c: Status 404 returned error can't find the container with id 94540cdda281434c4febf061b5f70816b5ee824f5f3f34280ee1f646796d571c Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.754718 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 22:32:10.835807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.758204 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:32:10.919497 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.848495 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:10.919497 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:10.857397 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51564124_a668_464c_80ef_2985c3c6c109.slice/crio-0ccd548094c00758b7ceb741bac9718722d20efb28ebad5450eb1f0649405d49 WatchSource:0}: Error finding container 0ccd548094c00758b7ceb741bac9718722d20efb28ebad5450eb1f0649405d49: Status 404 returned error can't find the container with id 0ccd548094c00758b7ceb741bac9718722d20efb28ebad5450eb1f0649405d49 Jul 01 22:32:10.919497 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.873558 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7krv" Jul 01 22:32:10.919497 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:10.890584 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e4ae6ed_cd79_44f8_b3ff_3abc01bc55e5.slice/crio-2592b628d5f56a02ced16a60653b47f136ec56333ed9badf69f81484bae3b654 WatchSource:0}: Error finding container 2592b628d5f56a02ced16a60653b47f136ec56333ed9badf69f81484bae3b654: Status 404 returned error can't find the container with id 2592b628d5f56a02ced16a60653b47f136ec56333ed9badf69f81484bae3b654 Jul 01 22:32:10.919497 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.898617 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c92kx" Jul 01 22:32:10.928276 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.926588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pxjt" Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.931525 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pswsl" Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:10.933869 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e5d35c_7bb6_4eda_b264_e25731987f58.slice/crio-6ad5b06f0da61592a09a041398669ff2963f282befe574f51dfa5b690500527d WatchSource:0}: Error finding container 6ad5b06f0da61592a09a041398669ff2963f282befe574f51dfa5b690500527d: Status 404 returned error can't find the container with id 6ad5b06f0da61592a09a041398669ff2963f282befe574f51dfa5b690500527d Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:10.952843 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c53760f_41eb_4097_8252_7002953475fc.slice/crio-becfd2626fa7da8aa958cab7d6198774f6b6e259bc64ec38229ea24e32c3aabf WatchSource:0}: Error finding container becfd2626fa7da8aa958cab7d6198774f6b6e259bc64ec38229ea24e32c3aabf: Status 404 returned error can't find the container with id becfd2626fa7da8aa958cab7d6198774f6b6e259bc64ec38229ea24e32c3aabf Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.965255 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zxcgk" Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.973974 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:10.974278 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32433b1_92f3_4ac7_ab9a_0ad2c373c576.slice/crio-f7fa1d6c386f442b9e8e0b5bea204cb00a7150804f8a124d763be87488d44a0e WatchSource:0}: Error finding container f7fa1d6c386f442b9e8e0b5bea204cb00a7150804f8a124d763be87488d44a0e: Status 404 returned error can't find the container with id f7fa1d6c386f442b9e8e0b5bea204cb00a7150804f8a124d763be87488d44a0e Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:10.981136 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f450d1f_663d_4db7_89f0_b2622c0eff7e.slice/crio-24bcef68f9b747465dde3b7f49c7dfdd7de7ba6b65304cccf6e0c7764fb8a9c3 WatchSource:0}: Error finding container 24bcef68f9b747465dde3b7f49c7dfdd7de7ba6b65304cccf6e0c7764fb8a9c3: Status 404 returned error can't find the container with id 24bcef68f9b747465dde3b7f49c7dfdd7de7ba6b65304cccf6e0c7764fb8a9c3 Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:10.996489 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.003109 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:11.006187 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b2f416_649a_4170_bd37_210de6580d47.slice/crio-4850d71dfd67f487ca6dbf6eb6f1d7b58f7b2a97e200fffa97f160b9d4f23095 WatchSource:0}: Error finding container 4850d71dfd67f487ca6dbf6eb6f1d7b58f7b2a97e200fffa97f160b9d4f23095: Status 404 returned error can't find the container with id 4850d71dfd67f487ca6dbf6eb6f1d7b58f7b2a97e200fffa97f160b9d4f23095 Jul 01 22:32:11.020759 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:11.010260 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31506d3f_0743_4684_8b09_aa952f816530.slice/crio-dcc55efbf3e1b19b5e29f39debf7d2cde1700c8f7fe2749ac1c46db71068da2b WatchSource:0}: Error finding container dcc55efbf3e1b19b5e29f39debf7d2cde1700c8f7fe2749ac1c46db71068da2b: Status 404 returned error can't find the container with id dcc55efbf3e1b19b5e29f39debf7d2cde1700c8f7fe2749ac1c46db71068da2b Jul 01 22:32:11.147978 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.125954 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:11.147978 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.126097 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:11.147978 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.126150 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs podName:a25f676f-13ae-48b5-b613-04af7f4f078e nodeName:}" failed. No retries permitted until 2026-07-01 22:32:12.126135961 +0000 UTC m=+3.013393897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs") pod "network-metrics-daemon-2dxrj" (UID: "a25f676f-13ae-48b5-b613-04af7f4f078e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:11.290506 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.227005 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:11.290506 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.227161 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:11.290506 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.227223 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret podName:889ac5e9-344c-4000-8257-66a0ef85e0eb nodeName:}" failed. No retries permitted until 2026-07-01 22:32:12.227203392 +0000 UTC m=+3.114461349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret") pod "global-pull-secret-syncer-4g9k9" (UID: "889ac5e9-344c-4000-8257-66a0ef85e0eb") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:11.328409 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.328368 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:11.328577 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.328559 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:32:11.328636 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.328588 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:32:11.328636 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.328602 2581 projected.go:196] Error preparing data for projected volume kube-api-access-c4df5 for pod openshift-network-diagnostics/network-check-target-gwf6l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:11.328755 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.328665 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5 podName:fd36e128-b67d-4e16-acdf-b755b5fdbbca nodeName:}" failed. No retries permitted until 2026-07-01 22:32:12.328646321 +0000 UTC m=+3.215904279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4df5" (UniqueName: "kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5") pod "network-check-target-gwf6l" (UID: "fd36e128-b67d-4e16-acdf-b755b5fdbbca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:11.661765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.661735 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:11.662610 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:11.661869 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:11.664792 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.664756 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 22:27:11 +0000 UTC" deadline="2028-01-02 14:21:33.271388206 +0000 UTC" Jul 01 22:32:11.664792 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.664792 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13191h49m21.606600388s" Jul 01 22:32:11.693610 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.693537 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lrs62" event={"ID":"51564124-a668-464c-80ef-2985c3c6c109","Type":"ContainerStarted","Data":"0ccd548094c00758b7ceb741bac9718722d20efb28ebad5450eb1f0649405d49"} Jul 01 22:32:11.713168 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.713141 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:32:11.720858 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.720821 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" event={"ID":"1a76c7746494e50a2be891e8cc385fe8","Type":"ContainerStarted","Data":"94540cdda281434c4febf061b5f70816b5ee824f5f3f34280ee1f646796d571c"} Jul 01 22:32:11.734109 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.734072 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" event={"ID":"88b2f416-649a-4170-bd37-210de6580d47","Type":"ContainerStarted","Data":"4850d71dfd67f487ca6dbf6eb6f1d7b58f7b2a97e200fffa97f160b9d4f23095"} Jul 01 22:32:11.740424 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.740394 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pswsl" event={"ID":"9c53760f-41eb-4097-8252-7002953475fc","Type":"ContainerStarted","Data":"becfd2626fa7da8aa958cab7d6198774f6b6e259bc64ec38229ea24e32c3aabf"} Jul 01 22:32:11.756657 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.756235 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" event={"ID":"1d6d3eb16eb91756b002dffb0f47321a","Type":"ContainerStarted","Data":"fe4411a13d819d045d063bc7cb31325182e88fe74a670df855880308e1803d5e"} Jul 01 22:32:11.775034 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.775000 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" event={"ID":"31506d3f-0743-4684-8b09-aa952f816530","Type":"ContainerStarted","Data":"dcc55efbf3e1b19b5e29f39debf7d2cde1700c8f7fe2749ac1c46db71068da2b"} Jul 01 22:32:11.799704 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.799524 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"24bcef68f9b747465dde3b7f49c7dfdd7de7ba6b65304cccf6e0c7764fb8a9c3"} Jul 01 22:32:11.812227 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.812110 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zxcgk" event={"ID":"b32433b1-92f3-4ac7-ab9a-0ad2c373c576","Type":"ContainerStarted","Data":"f7fa1d6c386f442b9e8e0b5bea204cb00a7150804f8a124d763be87488d44a0e"} Jul 01 22:32:11.815643 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.815613 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pxjt" event={"ID":"09e5d35c-7bb6-4eda-b264-e25731987f58","Type":"ContainerStarted","Data":"6ad5b06f0da61592a09a041398669ff2963f282befe574f51dfa5b690500527d"} Jul 01 22:32:11.830965 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.830860 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c92kx" event={"ID":"ba4ff25c-4df9-47ff-8c80-5e1a10867213","Type":"ContainerStarted","Data":"9afc5e354de7880c6fd9ba9c695abe9b8cac024743cfe6ab2a16bb0fa0e3bbdf"} Jul 01 22:32:11.836136 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:11.836093 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7krv" event={"ID":"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5","Type":"ContainerStarted","Data":"2592b628d5f56a02ced16a60653b47f136ec56333ed9badf69f81484bae3b654"} Jul 01 22:32:12.134392 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:12.134121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:12.134392 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.134254 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:12.134392 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.134329 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs podName:a25f676f-13ae-48b5-b613-04af7f4f078e nodeName:}" failed. No retries permitted until 2026-07-01 22:32:14.134309525 +0000 UTC m=+5.021567476 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs") pod "network-metrics-daemon-2dxrj" (UID: "a25f676f-13ae-48b5-b613-04af7f4f078e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:12.235958 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:12.235324 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:12.235958 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.235474 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:12.235958 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.235536 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret podName:889ac5e9-344c-4000-8257-66a0ef85e0eb nodeName:}" failed. No retries permitted until 2026-07-01 22:32:14.235518721 +0000 UTC m=+5.122776679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret") pod "global-pull-secret-syncer-4g9k9" (UID: "889ac5e9-344c-4000-8257-66a0ef85e0eb") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:12.292797 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:12.292764 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:32:12.335921 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:12.335884 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:12.336098 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.336058 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:32:12.336098 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.336078 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:32:12.336256 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.336109 2581 projected.go:196] Error preparing data for projected volume kube-api-access-c4df5 for pod openshift-network-diagnostics/network-check-target-gwf6l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:12.336256 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.336168 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5 podName:fd36e128-b67d-4e16-acdf-b755b5fdbbca nodeName:}" failed. No retries permitted until 2026-07-01 22:32:14.336150223 +0000 UTC m=+5.223408163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4df5" (UniqueName: "kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5") pod "network-check-target-gwf6l" (UID: "fd36e128-b67d-4e16-acdf-b755b5fdbbca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:12.660053 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:12.659384 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:12.660053 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.659519 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:12.660053 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:12.659604 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:12.660053 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:12.659686 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:12.665562 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:12.665416 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 22:27:11 +0000 UTC" deadline="2028-01-28 22:11:16.527502609 +0000 UTC" Jul 01 22:32:12.665562 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:12.665471 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13823h39m3.862035865s" Jul 01 22:32:13.118464 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:13.118383 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:32:13.671153 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:13.671101 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:13.671628 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:13.671266 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:14.152112 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:14.152050 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:14.152345 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.152300 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:14.152441 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.152382 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs podName:a25f676f-13ae-48b5-b613-04af7f4f078e nodeName:}" failed. No retries permitted until 2026-07-01 22:32:18.152358628 +0000 UTC m=+9.039616567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs") pod "network-metrics-daemon-2dxrj" (UID: "a25f676f-13ae-48b5-b613-04af7f4f078e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:14.253667 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:14.253577 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:14.253839 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.253765 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:14.253839 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.253832 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret podName:889ac5e9-344c-4000-8257-66a0ef85e0eb nodeName:}" failed. No retries permitted until 2026-07-01 22:32:18.253812669 +0000 UTC m=+9.141070608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret") pod "global-pull-secret-syncer-4g9k9" (UID: "889ac5e9-344c-4000-8257-66a0ef85e0eb") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:14.355230 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:14.354538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:14.355230 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.354730 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:32:14.355230 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.354752 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:32:14.355230 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.354767 2581 projected.go:196] Error preparing data for projected volume kube-api-access-c4df5 for pod openshift-network-diagnostics/network-check-target-gwf6l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:14.355230 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.354831 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5 podName:fd36e128-b67d-4e16-acdf-b755b5fdbbca nodeName:}" failed. No retries permitted until 2026-07-01 22:32:18.354811843 +0000 UTC m=+9.242069799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4df5" (UniqueName: "kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5") pod "network-check-target-gwf6l" (UID: "fd36e128-b67d-4e16-acdf-b755b5fdbbca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:14.659830 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:14.659404 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:14.659830 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.659559 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:14.659830 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:14.659404 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:14.660255 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:14.659983 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:15.659449 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:15.659415 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:15.660110 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:15.660080 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:16.659346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:16.659235 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:16.659346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:16.659261 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:16.659924 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:16.659386 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:16.659924 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:16.659526 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:17.659410 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:17.659371 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:17.659583 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:17.659523 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:18.187337 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:18.186637 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:18.187337 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.186852 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:18.187337 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.186935 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs podName:a25f676f-13ae-48b5-b613-04af7f4f078e nodeName:}" failed. No retries permitted until 2026-07-01 22:32:26.186905092 +0000 UTC m=+17.074163031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs") pod "network-metrics-daemon-2dxrj" (UID: "a25f676f-13ae-48b5-b613-04af7f4f078e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:18.287694 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:18.287600 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:18.287891 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.287784 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:18.287891 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.287874 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret podName:889ac5e9-344c-4000-8257-66a0ef85e0eb nodeName:}" failed. No retries permitted until 2026-07-01 22:32:26.287851431 +0000 UTC m=+17.175109385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret") pod "global-pull-secret-syncer-4g9k9" (UID: "889ac5e9-344c-4000-8257-66a0ef85e0eb") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:18.389074 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:18.389034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:18.389270 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.389237 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:32:18.389270 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.389258 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:32:18.389270 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.389272 2581 projected.go:196] Error preparing data for projected volume kube-api-access-c4df5 for pod openshift-network-diagnostics/network-check-target-gwf6l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:18.389462 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.389346 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5 podName:fd36e128-b67d-4e16-acdf-b755b5fdbbca nodeName:}" failed. No retries permitted until 2026-07-01 22:32:26.38932779 +0000 UTC m=+17.276585745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4df5" (UniqueName: "kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5") pod "network-check-target-gwf6l" (UID: "fd36e128-b67d-4e16-acdf-b755b5fdbbca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:18.659527 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:18.659350 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:18.659527 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:18.659368 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:18.659527 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.659478 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:18.660077 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:18.659589 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:19.660557 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:19.660153 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:19.660557 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:19.660260 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:20.659701 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:20.659192 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:20.659701 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:20.659224 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:20.659701 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:20.659332 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:20.659701 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:20.659499 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:21.659895 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:21.659848 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:21.660435 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:21.659993 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:22.659797 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:22.659765 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:22.659797 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:22.659793 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:22.660308 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:22.659880 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:22.660308 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:22.660021 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:23.659686 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:23.659652 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:23.659816 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:23.659781 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:24.659426 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:24.659396 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:24.659864 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:24.659396 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:24.659864 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:24.659530 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:24.659864 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:24.659602 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:25.659609 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:25.659577 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:25.660046 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:25.659716 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:26.251442 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:26.251355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:26.251661 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.251483 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:26.251661 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.251544 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs podName:a25f676f-13ae-48b5-b613-04af7f4f078e nodeName:}" failed. No retries permitted until 2026-07-01 22:32:42.251525199 +0000 UTC m=+33.138783156 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs") pod "network-metrics-daemon-2dxrj" (UID: "a25f676f-13ae-48b5-b613-04af7f4f078e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:26.352401 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:26.352350 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:26.352593 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.352514 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:26.352593 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.352593 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret podName:889ac5e9-344c-4000-8257-66a0ef85e0eb nodeName:}" failed. No retries permitted until 2026-07-01 22:32:42.352570903 +0000 UTC m=+33.239828841 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret") pod "global-pull-secret-syncer-4g9k9" (UID: "889ac5e9-344c-4000-8257-66a0ef85e0eb") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:26.453521 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:26.453457 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:26.453663 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.453644 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:32:26.453704 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.453670 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:32:26.453704 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.453683 2581 projected.go:196] Error preparing data for projected volume kube-api-access-c4df5 for pod openshift-network-diagnostics/network-check-target-gwf6l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:26.453842 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.453753 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5 podName:fd36e128-b67d-4e16-acdf-b755b5fdbbca nodeName:}" failed. No retries permitted until 2026-07-01 22:32:42.453738779 +0000 UTC m=+33.340996714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4df5" (UniqueName: "kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5") pod "network-check-target-gwf6l" (UID: "fd36e128-b67d-4e16-acdf-b755b5fdbbca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:26.658777 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:26.658709 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:26.658937 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.658804 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:26.658937 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:26.658860 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:26.659067 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:26.658968 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:27.658841 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:27.658801 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:27.659333 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:27.658938 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:28.658964 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:28.658928 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:28.659450 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:28.658934 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:28.659450 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:28.659063 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:28.659450 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:28.659151 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:29.659994 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:29.659960 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:29.660491 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:29.660101 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:30.659067 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:30.659020 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:30.659252 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:30.659091 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:30.659252 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:30.659215 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:30.659372 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:30.659344 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:31.659702 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:31.659665 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:31.660131 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:31.659772 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:32.659803 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.659639 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:32.660424 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.659641 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:32.660424 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:32.659918 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:32.660424 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:32.659979 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:32.893196 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.893160 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pswsl" event={"ID":"9c53760f-41eb-4097-8252-7002953475fc","Type":"ContainerStarted","Data":"166cacdb22b47002a2e69f9a11380aaca4862a92a4534b64a50c174629d07066"} Jul 01 22:32:32.894613 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.894586 2581 generic.go:356] "Generic (PLEG): container finished" podID="1d6d3eb16eb91756b002dffb0f47321a" containerID="1aa1497a286aed253ddd9886d7dd0279dcc93114cb2967b9f4e65bd0649b9238" exitCode=0 Jul 01 22:32:32.894740 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.894662 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" event={"ID":"1d6d3eb16eb91756b002dffb0f47321a","Type":"ContainerDied","Data":"1aa1497a286aed253ddd9886d7dd0279dcc93114cb2967b9f4e65bd0649b9238"} Jul 01 22:32:32.896265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.896244 2581 generic.go:356] "Generic (PLEG): container finished" podID="31506d3f-0743-4684-8b09-aa952f816530" containerID="eed330d0f9c1e18f9174971527c7215b2f46eb9873891d8cf9e3afc570d9de73" exitCode=0 Jul 01 22:32:32.896365 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.896327 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" event={"ID":"31506d3f-0743-4684-8b09-aa952f816530","Type":"ContainerDied","Data":"eed330d0f9c1e18f9174971527c7215b2f46eb9873891d8cf9e3afc570d9de73"} Jul 01 22:32:32.899430 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.899410 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"01748f566f34558098f490d029cea3770e2ba07ef7e703664dd4f4050e20d5c2"} Jul 01 22:32:32.899531 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.899474 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"f3716bd9431ada25011bdca078c31adcea7c250f85783b5f157ace55bd4ad731"} Jul 01 22:32:32.899531 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.899514 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"1e52474d70b7caeeb11b5b5d55ab4cf35ff4625ecb2e116a98fdb59790ba740c"} Jul 01 22:32:32.899661 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.899565 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"a00b15e391adab5d47a5aa98a8e61b4a76429197fb5c1d1f8f57972c78f509c4"} Jul 01 22:32:32.899661 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.899597 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"34343df6ca2aab748969bc0e4f86854fbc5175f12640dcf7b0337d5ce0e0b579"} Jul 01 22:32:32.899661 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.899631 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"eff3de20963df33954a14b4a349352ca17c468f465d58d1c3f9da205b4a3dfb9"} Jul 01 22:32:32.900742 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.900717 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pxjt" event={"ID":"09e5d35c-7bb6-4eda-b264-e25731987f58","Type":"ContainerStarted","Data":"e6c0e06123d3143939549c889bea447cb28fc1683606c5bba62d8d275ce4e2bb"} Jul 01 22:32:32.902027 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.902007 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c92kx" event={"ID":"ba4ff25c-4df9-47ff-8c80-5e1a10867213","Type":"ContainerStarted","Data":"d5ff1eaabcc369de759b26403c667d58273f32e17097daa10cc873ff91a6c665"} Jul 01 22:32:32.903403 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.903383 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7krv" event={"ID":"1e4ae6ed-cd79-44f8-b3ff-3abc01bc55e5","Type":"ContainerStarted","Data":"711d80d480387aa2637dff2a8d9f44b5d186a06c4115a07304deb8f54fd7173d"} Jul 01 22:32:32.904643 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.904622 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lrs62" event={"ID":"51564124-a668-464c-80ef-2985c3c6c109","Type":"ContainerStarted","Data":"d7ea3e2c9312254ed3b6eb96add4b553274a4ceac75a8b0fdb734aed12de0815"} Jul 01 22:32:32.905885 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.905868 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" event={"ID":"1a76c7746494e50a2be891e8cc385fe8","Type":"ContainerStarted","Data":"69cdf47531302632047dbfedffa6c64ca208b82d15ba607921a5ca91f7c2c75e"} Jul 01 22:32:32.907093 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.907074 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" event={"ID":"88b2f416-649a-4170-bd37-210de6580d47","Type":"ContainerStarted","Data":"a074f0501e1d7d90cf60dbdbcf3e1ec8ddb384e37e48ce8ead545966c1c5e957"} Jul 01 22:32:32.914465 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.914428 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pswsl" podStartSLOduration=1.769577663 podStartE2EDuration="22.914418357s" totalImagesPullingTime="21.144840694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:32.914358516 +0000 UTC m=+23.801616474" watchObservedRunningTime="2026-07-01 22:32:32.914418357 +0000 UTC m=+23.801676307" Jul 01 22:32:32.929299 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.929266 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-190.ec2.internal" podStartSLOduration=21.929255151 podStartE2EDuration="21.929255151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:32.928801298 +0000 UTC m=+23.816059257" watchObservedRunningTime="2026-07-01 22:32:32.929255151 +0000 UTC m=+23.816513108" Jul 01 22:32:32.946067 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.945991 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lrs62" podStartSLOduration=1.769764104 podStartE2EDuration="22.945983461s" totalImagesPullingTime="21.176219357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:32.945500869 +0000 UTC m=+23.832758827" watchObservedRunningTime="2026-07-01 22:32:32.945983461 +0000 UTC m=+23.833241417" Jul 01 22:32:32.966604 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.966571 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m7krv" podStartSLOduration=1.805276723 podStartE2EDuration="22.966559979s" totalImagesPullingTime="21.161283256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:32.966538698 +0000 UTC m=+23.853796655" watchObservedRunningTime="2026-07-01 22:32:32.966559979 +0000 UTC m=+23.853817935" Jul 01 22:32:32.983018 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:32.982983 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-c92kx" podStartSLOduration=1.8574668060000001 podStartE2EDuration="22.982970631s" totalImagesPullingTime="21.125503825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:32.982838401 +0000 UTC m=+23.870096360" watchObservedRunningTime="2026-07-01 22:32:32.982970631 +0000 UTC m=+23.870228588" Jul 01 22:32:33.021789 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.021556 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2pxjt" podStartSLOduration=1.9200813509999999 podStartE2EDuration="23.021537463s" totalImagesPullingTime="21.101456112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:33.021279351 +0000 UTC m=+23.908537309" watchObservedRunningTime="2026-07-01 22:32:33.021537463 +0000 UTC m=+23.908795421" Jul 01 22:32:33.342406 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.342361 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:33.343008 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.342987 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:33.659202 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.659121 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:33.659374 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:33.659250 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:33.912065 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.911732 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" event={"ID":"1d6d3eb16eb91756b002dffb0f47321a","Type":"ContainerStarted","Data":"7b08993f0acfe66e85412f7205b85bf2780e0affaa617c8c53af364abf0b6053"} Jul 01 22:32:33.913567 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.913515 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zxcgk" event={"ID":"b32433b1-92f3-4ac7-ab9a-0ad2c373c576","Type":"ContainerStarted","Data":"73bcb2df86886e88be1399a8a8cd1dbea5fdc551d7e8559d3fdc1f7f17e21f92"} Jul 01 22:32:33.914064 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.913858 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:33.914807 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.914784 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lrs62" Jul 01 22:32:33.929074 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.929011 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-190.ec2.internal" podStartSLOduration=22.92899342 podStartE2EDuration="22.92899342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:33.928486018 +0000 UTC m=+24.815743975" watchObservedRunningTime="2026-07-01 22:32:33.92899342 +0000 UTC m=+24.816251377" Jul 01 22:32:33.946329 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:33.946286 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zxcgk" podStartSLOduration=2.878058205 podStartE2EDuration="23.94627215s" totalImagesPullingTime="21.068213945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:33.945750554 +0000 UTC m=+24.833008513" watchObservedRunningTime="2026-07-01 22:32:33.94627215 +0000 UTC m=+24.833530106" Jul 01 22:32:34.020774 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:34.020432 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 22:32:34.633629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:34.633503 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T22:32:34.020445584Z","UUID":"31916cb6-43b6-4dca-9cc8-1d45c04b9897","Handler":null,"Name":"","Endpoint":""} Jul 01 22:32:34.635335 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:34.635311 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 22:32:34.635471 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:34.635359 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 22:32:34.659246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:34.659218 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:34.659399 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:34.659338 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:34.659464 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:34.659407 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:34.659528 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:34.659509 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:34.918253 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:34.918120 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" event={"ID":"88b2f416-649a-4170-bd37-210de6580d47","Type":"ContainerStarted","Data":"42215314935e829f3431573d4896e8614b375eb1f6a2f08f566f87444ad115c8"} Jul 01 22:32:35.658744 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:35.658668 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:35.658897 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:35.658807 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:35.924237 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:35.924153 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"f9d703a8f5692ba16d1d78be6098c1a5a4cafd6c5e7edfbcc65eaf644b0db3ad"} Jul 01 22:32:35.926345 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:35.926319 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" event={"ID":"88b2f416-649a-4170-bd37-210de6580d47","Type":"ContainerStarted","Data":"4c670f4a41a01801821776148db0ef439347c1d396e04874005ca2be9a76e0a3"} Jul 01 22:32:35.946876 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:35.946825 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dzdtz" podStartSLOduration=1.434197177 podStartE2EDuration="24.946811967s" totalImagesPullingTime="23.51261479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:35.946757835 +0000 UTC m=+26.834015791" watchObservedRunningTime="2026-07-01 22:32:35.946811967 +0000 UTC m=+26.834069907" Jul 01 22:32:36.659755 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:36.659721 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:36.659755 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:36.659739 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:36.659977 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:36.659866 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:36.660029 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:36.659976 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:37.659307 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:37.659134 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:37.659855 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:37.659377 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:37.932640 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:37.932607 2581 generic.go:356] "Generic (PLEG): container finished" podID="31506d3f-0743-4684-8b09-aa952f816530" containerID="35fe9a85a55c4dfdfcf2f5946560b271a3ead81e7c8da1ebc36916677cad2411" exitCode=0 Jul 01 22:32:37.932834 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:37.932687 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" event={"ID":"31506d3f-0743-4684-8b09-aa952f816530","Type":"ContainerDied","Data":"35fe9a85a55c4dfdfcf2f5946560b271a3ead81e7c8da1ebc36916677cad2411"} Jul 01 22:32:37.936478 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:37.936457 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" event={"ID":"8f450d1f-663d-4db7-89f0-b2622c0eff7e","Type":"ContainerStarted","Data":"04c4db42040f114e74f26d5da7004166a3c2e540b71000c399af90b2180904ff"} Jul 01 22:32:37.936689 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:37.936677 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:37.936740 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:37.936734 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:37.951580 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:37.951560 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:37.993554 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:37.993509 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" podStartSLOduration=5.869232952 podStartE2EDuration="26.993495334s" totalImagesPullingTime="21.124262382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:37.993091998 +0000 UTC m=+28.880349955" watchObservedRunningTime="2026-07-01 22:32:37.993495334 +0000 UTC m=+28.880753290" Jul 01 22:32:38.659008 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:38.658979 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:38.659290 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:38.659123 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:38.659290 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:38.659169 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:38.659290 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:38.659259 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:38.939271 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:38.939206 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:38.953563 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:38.953543 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:32:39.659916 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:39.659885 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:39.660376 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:39.659982 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:39.942973 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:39.942883 2581 generic.go:356] "Generic (PLEG): container finished" podID="31506d3f-0743-4684-8b09-aa952f816530" containerID="6bc8d2a4233d9ae2cbf89d4045424d509343e1a98fae6208fb3e36dc8dec9642" exitCode=0 Jul 01 22:32:39.943153 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:39.942968 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" event={"ID":"31506d3f-0743-4684-8b09-aa952f816530","Type":"ContainerDied","Data":"6bc8d2a4233d9ae2cbf89d4045424d509343e1a98fae6208fb3e36dc8dec9642"} Jul 01 22:32:40.659457 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:40.659423 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:40.659674 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:40.659423 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:40.659674 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:40.659521 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:40.659674 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:40.659596 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:41.659232 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:41.659017 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:41.659610 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:41.659318 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:41.949876 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:41.949800 2581 generic.go:356] "Generic (PLEG): container finished" podID="31506d3f-0743-4684-8b09-aa952f816530" containerID="5d1e226063c2b5e2bdede6f97a7de8d1234342a1214ca16c5522e6f96559c9e7" exitCode=0 Jul 01 22:32:41.949876 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:41.949849 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" event={"ID":"31506d3f-0743-4684-8b09-aa952f816530","Type":"ContainerDied","Data":"5d1e226063c2b5e2bdede6f97a7de8d1234342a1214ca16c5522e6f96559c9e7"} Jul 01 22:32:42.270207 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:42.270176 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:42.270383 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.270305 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:42.270383 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.270359 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs podName:a25f676f-13ae-48b5-b613-04af7f4f078e nodeName:}" failed. No retries permitted until 2026-07-01 22:33:14.270345643 +0000 UTC m=+65.157603579 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs") pod "network-metrics-daemon-2dxrj" (UID: "a25f676f-13ae-48b5-b613-04af7f4f078e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:32:42.371196 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:42.371161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:42.371357 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.371295 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:42.371357 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.371352 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret podName:889ac5e9-344c-4000-8257-66a0ef85e0eb nodeName:}" failed. No retries permitted until 2026-07-01 22:33:14.371337777 +0000 UTC m=+65.258595715 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret") pod "global-pull-secret-syncer-4g9k9" (UID: "889ac5e9-344c-4000-8257-66a0ef85e0eb") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:32:42.471648 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:42.471612 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:42.471829 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.471800 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:32:42.471829 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.471825 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:32:42.471941 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.471838 2581 projected.go:196] Error preparing data for projected volume kube-api-access-c4df5 for pod openshift-network-diagnostics/network-check-target-gwf6l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:42.471941 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.471895 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5 podName:fd36e128-b67d-4e16-acdf-b755b5fdbbca nodeName:}" failed. No retries permitted until 2026-07-01 22:33:14.471882085 +0000 UTC m=+65.359140019 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4df5" (UniqueName: "kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5") pod "network-check-target-gwf6l" (UID: "fd36e128-b67d-4e16-acdf-b755b5fdbbca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:32:42.659032 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:42.658939 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:42.659222 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:42.658940 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:42.659222 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.659089 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:42.659222 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:42.659151 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:43.659693 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:43.659653 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:43.660211 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:43.659784 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:44.545503 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:44.545467 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gwf6l"] Jul 01 22:32:44.545697 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:44.545642 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:44.545779 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:44.545750 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:44.548578 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:44.548532 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2dxrj"] Jul 01 22:32:44.548730 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:44.548683 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:44.548869 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:44.548842 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:44.549296 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:44.549256 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4g9k9"] Jul 01 22:32:44.549370 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:44.549338 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:44.549449 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:44.549429 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:46.659541 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:46.659505 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:46.659884 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:46.659511 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:46.659884 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:46.659614 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:46.659884 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:46.659521 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:46.659884 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:46.659690 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:46.659884 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:46.659742 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:46.967507 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:46.967476 2581 generic.go:356] "Generic (PLEG): container finished" podID="31506d3f-0743-4684-8b09-aa952f816530" containerID="9aa0eeb18d4915bf0c00c4cb1963710e2ac81b5b49cfa8f054b901194cb79fde" exitCode=0 Jul 01 22:32:46.967672 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:46.967514 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" event={"ID":"31506d3f-0743-4684-8b09-aa952f816530","Type":"ContainerDied","Data":"9aa0eeb18d4915bf0c00c4cb1963710e2ac81b5b49cfa8f054b901194cb79fde"} Jul 01 22:32:47.973661 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:47.973414 2581 generic.go:356] "Generic (PLEG): container finished" podID="31506d3f-0743-4684-8b09-aa952f816530" containerID="f9ad94b2d03f662c3b7494a9eaf12390ca284932cd85d75bd958dbd46ab2ff99" exitCode=0 Jul 01 22:32:47.973661 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:47.973491 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" event={"ID":"31506d3f-0743-4684-8b09-aa952f816530","Type":"ContainerDied","Data":"f9ad94b2d03f662c3b7494a9eaf12390ca284932cd85d75bd958dbd46ab2ff99"} Jul 01 22:32:48.659697 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.659646 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:48.659846 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.659646 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:48.659880 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.659646 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:48.659944 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:48.659763 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwf6l" podUID="fd36e128-b67d-4e16-acdf-b755b5fdbbca" Jul 01 22:32:48.659981 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:48.659937 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4g9k9" podUID="889ac5e9-344c-4000-8257-66a0ef85e0eb" Jul 01 22:32:48.659981 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:48.659854 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dxrj" podUID="a25f676f-13ae-48b5-b613-04af7f4f078e" Jul 01 22:32:48.913367 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.913291 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-190.ec2.internal" event="NodeReady" Jul 01 22:32:48.913502 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.913431 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 22:32:48.972325 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.972297 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rwvtw"] Jul 01 22:32:48.974266 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.974241 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:48.974823 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.974604 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xm7xn"] Jul 01 22:32:48.977035 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.977016 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xm7xn" Jul 01 22:32:48.977630 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.977609 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 22:32:48.977630 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.977620 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 22:32:48.977778 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.977705 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6t6vb\"" Jul 01 22:32:48.981215 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.980170 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 22:32:48.981215 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.980419 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j9ggs\"" Jul 01 22:32:48.981215 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.980492 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 22:32:48.981215 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.980637 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 22:32:48.983219 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.983191 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" event={"ID":"31506d3f-0743-4684-8b09-aa952f816530","Type":"ContainerStarted","Data":"17c6d5ae71787fe795502a2cfa18f982ed2866d567e10e726270d940f6e60794"} Jul 01 22:32:48.988759 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:48.988736 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rwvtw"] Jul 01 22:32:49.008561 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.008534 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xm7xn"] Jul 01 22:32:49.080655 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.080611 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fhm8h" podStartSLOduration=6.209009259 podStartE2EDuration="38.080595466s" totalImagesPullingTime="31.871586207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:49.079519691 +0000 UTC m=+39.966777641" watchObservedRunningTime="2026-07-01 22:32:49.080595466 +0000 UTC m=+39.967853423" Jul 01 22:32:49.086224 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.086202 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4f4kd"] Jul 01 22:32:49.088634 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.088620 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.091795 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.091774 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 22:32:49.092007 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.091991 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 22:32:49.092104 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.092071 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x5v5q\"" Jul 01 22:32:49.092287 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.092272 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 22:32:49.092368 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.092345 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 22:32:49.107152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.107129 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4f4kd"] Jul 01 22:32:49.117958 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.117934 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4eaf10c2-65d3-44bb-a063-2011878aba2b-config-volume\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.118039 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.117980 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4eaf10c2-65d3-44bb-a063-2011878aba2b-metrics-tls\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.118039 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.118007 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4eaf10c2-65d3-44bb-a063-2011878aba2b-tmp-dir\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.118171 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.118065 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wd62\" (UniqueName: \"kubernetes.io/projected/4eaf10c2-65d3-44bb-a063-2011878aba2b-kube-api-access-5wd62\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.118218 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.118164 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frcmb\" (UniqueName: \"kubernetes.io/projected/6aa2539a-b2b3-4e99-a5d2-13c56f919590-kube-api-access-frcmb\") pod \"ingress-canary-xm7xn\" (UID: \"6aa2539a-b2b3-4e99-a5d2-13c56f919590\") " pod="openshift-ingress-canary/ingress-canary-xm7xn" Jul 01 22:32:49.118274 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.118255 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6aa2539a-b2b3-4e99-a5d2-13c56f919590-cert\") pod \"ingress-canary-xm7xn\" (UID: \"6aa2539a-b2b3-4e99-a5d2-13c56f919590\") " pod="openshift-ingress-canary/ingress-canary-xm7xn" Jul 01 22:32:49.219139 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219109 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frcmb\" (UniqueName: \"kubernetes.io/projected/6aa2539a-b2b3-4e99-a5d2-13c56f919590-kube-api-access-frcmb\") pod \"ingress-canary-xm7xn\" (UID: \"6aa2539a-b2b3-4e99-a5d2-13c56f919590\") " pod="openshift-ingress-canary/ingress-canary-xm7xn" Jul 01 22:32:49.219287 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6aa2539a-b2b3-4e99-a5d2-13c56f919590-cert\") pod \"ingress-canary-xm7xn\" (UID: \"6aa2539a-b2b3-4e99-a5d2-13c56f919590\") " pod="openshift-ingress-canary/ingress-canary-xm7xn" Jul 01 22:32:49.219287 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219203 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4eaf10c2-65d3-44bb-a063-2011878aba2b-config-volume\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.219287 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219225 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ae01200-de56-4195-a919-151c0a717eed-crio-socket\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.219287 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219259 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbvkb\" (UniqueName: \"kubernetes.io/projected/1ae01200-de56-4195-a919-151c0a717eed-kube-api-access-rbvkb\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.219453 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219290 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ae01200-de56-4195-a919-151c0a717eed-data-volume\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.219453 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219320 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4eaf10c2-65d3-44bb-a063-2011878aba2b-metrics-tls\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.219453 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219344 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ae01200-de56-4195-a919-151c0a717eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.219453 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ae01200-de56-4195-a919-151c0a717eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.219453 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4eaf10c2-65d3-44bb-a063-2011878aba2b-tmp-dir\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.219453 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219425 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wd62\" (UniqueName: \"kubernetes.io/projected/4eaf10c2-65d3-44bb-a063-2011878aba2b-kube-api-access-5wd62\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.219706 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219685 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4eaf10c2-65d3-44bb-a063-2011878aba2b-tmp-dir\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.219908 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.219886 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4eaf10c2-65d3-44bb-a063-2011878aba2b-config-volume\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.222957 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.222933 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4eaf10c2-65d3-44bb-a063-2011878aba2b-metrics-tls\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.223164 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.223144 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6aa2539a-b2b3-4e99-a5d2-13c56f919590-cert\") pod \"ingress-canary-xm7xn\" (UID: \"6aa2539a-b2b3-4e99-a5d2-13c56f919590\") " pod="openshift-ingress-canary/ingress-canary-xm7xn" Jul 01 22:32:49.228256 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.228234 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wd62\" (UniqueName: \"kubernetes.io/projected/4eaf10c2-65d3-44bb-a063-2011878aba2b-kube-api-access-5wd62\") pod \"dns-default-rwvtw\" (UID: \"4eaf10c2-65d3-44bb-a063-2011878aba2b\") " pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.228361 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.228342 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frcmb\" (UniqueName: \"kubernetes.io/projected/6aa2539a-b2b3-4e99-a5d2-13c56f919590-kube-api-access-frcmb\") pod \"ingress-canary-xm7xn\" (UID: \"6aa2539a-b2b3-4e99-a5d2-13c56f919590\") " pod="openshift-ingress-canary/ingress-canary-xm7xn" Jul 01 22:32:49.287330 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.287299 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:49.292653 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.292635 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xm7xn" Jul 01 22:32:49.320153 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.320121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ae01200-de56-4195-a919-151c0a717eed-crio-socket\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.320313 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.320166 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbvkb\" (UniqueName: \"kubernetes.io/projected/1ae01200-de56-4195-a919-151c0a717eed-kube-api-access-rbvkb\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.320313 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.320197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ae01200-de56-4195-a919-151c0a717eed-data-volume\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.320313 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.320222 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ae01200-de56-4195-a919-151c0a717eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.320313 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.320247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ae01200-de56-4195-a919-151c0a717eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.320505 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.320355 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ae01200-de56-4195-a919-151c0a717eed-crio-socket\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.320613 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.320592 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ae01200-de56-4195-a919-151c0a717eed-data-volume\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.320808 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.320790 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ae01200-de56-4195-a919-151c0a717eed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.322524 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.322504 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ae01200-de56-4195-a919-151c0a717eed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.336946 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.336920 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbvkb\" (UniqueName: \"kubernetes.io/projected/1ae01200-de56-4195-a919-151c0a717eed-kube-api-access-rbvkb\") pod \"insights-runtime-extractor-4f4kd\" (UID: \"1ae01200-de56-4195-a919-151c0a717eed\") " pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.396669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.396644 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4f4kd" Jul 01 22:32:49.443702 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.443671 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xm7xn"] Jul 01 22:32:49.446819 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:49.446792 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa2539a_b2b3_4e99_a5d2_13c56f919590.slice/crio-a0ef9e95141856f78c5b32c16a827363812948254d8f455db92b032785bff1f9 WatchSource:0}: Error finding container a0ef9e95141856f78c5b32c16a827363812948254d8f455db92b032785bff1f9: Status 404 returned error can't find the container with id a0ef9e95141856f78c5b32c16a827363812948254d8f455db92b032785bff1f9 Jul 01 22:32:49.465096 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.465067 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rwvtw"] Jul 01 22:32:49.467951 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:49.467925 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eaf10c2_65d3_44bb_a063_2011878aba2b.slice/crio-bdcb724e7c4845c6b80518a21401aa130f4df09716093c1f90bdd788dcbc591c WatchSource:0}: Error finding container bdcb724e7c4845c6b80518a21401aa130f4df09716093c1f90bdd788dcbc591c: Status 404 returned error can't find the container with id bdcb724e7c4845c6b80518a21401aa130f4df09716093c1f90bdd788dcbc591c Jul 01 22:32:49.535576 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.535545 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4f4kd"] Jul 01 22:32:49.538336 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:49.538311 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae01200_de56_4195_a919_151c0a717eed.slice/crio-8c8449601c6213e504e236988e519dce93918a7bd7fcf34171122fbb49f31780 WatchSource:0}: Error finding container 8c8449601c6213e504e236988e519dce93918a7bd7fcf34171122fbb49f31780: Status 404 returned error can't find the container with id 8c8449601c6213e504e236988e519dce93918a7bd7fcf34171122fbb49f31780 Jul 01 22:32:49.989189 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.988822 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4f4kd" event={"ID":"1ae01200-de56-4195-a919-151c0a717eed","Type":"ContainerStarted","Data":"8c8449601c6213e504e236988e519dce93918a7bd7fcf34171122fbb49f31780"} Jul 01 22:32:49.991230 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.991198 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xm7xn" event={"ID":"6aa2539a-b2b3-4e99-a5d2-13c56f919590","Type":"ContainerStarted","Data":"a0ef9e95141856f78c5b32c16a827363812948254d8f455db92b032785bff1f9"} Jul 01 22:32:49.993973 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:49.993944 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rwvtw" event={"ID":"4eaf10c2-65d3-44bb-a063-2011878aba2b","Type":"ContainerStarted","Data":"bdcb724e7c4845c6b80518a21401aa130f4df09716093c1f90bdd788dcbc591c"} Jul 01 22:32:50.159141 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.159106 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-fkz95"] Jul 01 22:32:50.177035 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.176951 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.181301 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.179370 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-fkz95"] Jul 01 22:32:50.181977 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.181953 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 22:32:50.182296 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.182278 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 22:32:50.182482 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.182467 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 22:32:50.182699 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.182682 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 01 22:32:50.182867 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.182853 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 01 22:32:50.183992 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.183971 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xwnvk\"" Jul 01 22:32:50.328032 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.327947 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d27g7\" (UniqueName: \"kubernetes.io/projected/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-kube-api-access-d27g7\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.328216 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.328068 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.328216 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.328117 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.328216 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.328154 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.428956 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.428920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.429162 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.428980 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.429162 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.429039 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d27g7\" (UniqueName: \"kubernetes.io/projected/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-kube-api-access-d27g7\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.429162 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.429129 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.430486 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.430420 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.433148 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.433124 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.435704 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.435658 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.459369 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.459342 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d27g7\" (UniqueName: \"kubernetes.io/projected/f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06-kube-api-access-d27g7\") pod \"prometheus-operator-7b7d896497-fkz95\" (UID: \"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.494473 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.494442 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" Jul 01 22:32:50.659824 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.659109 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:32:50.659824 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.659533 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:32:50.660029 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.659846 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:32:50.663026 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.663001 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7csn6\"" Jul 01 22:32:50.663874 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.663857 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 22:32:50.664790 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.664690 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 22:32:50.665234 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.664901 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 22:32:50.666360 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.666294 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 22:32:50.666675 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.666601 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v2q69\"" Jul 01 22:32:50.706350 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.706270 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-fkz95"] Jul 01 22:32:50.712170 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:50.712143 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6386ca7_3683_4f9a_bb84_7b0bc0eb8d06.slice/crio-d7220d1e775d3f38ca4451ead28fc35e318d585e748baf82c927732fdf54f243 WatchSource:0}: Error finding container d7220d1e775d3f38ca4451ead28fc35e318d585e748baf82c927732fdf54f243: Status 404 returned error can't find the container with id d7220d1e775d3f38ca4451ead28fc35e318d585e748baf82c927732fdf54f243 Jul 01 22:32:50.998665 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:50.998616 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" event={"ID":"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06","Type":"ContainerStarted","Data":"d7220d1e775d3f38ca4451ead28fc35e318d585e748baf82c927732fdf54f243"} Jul 01 22:32:56.303788 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.303762 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b8c776c7f-v7cqr"] Jul 01 22:32:56.316537 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.316519 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.319654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.319634 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 01 22:32:56.321789 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.321758 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 01 22:32:56.321888 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.321787 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 01 22:32:56.321888 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.321758 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 01 22:32:56.323112 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.322844 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fkgpw\"" Jul 01 22:32:56.323112 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.322926 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 01 22:32:56.323112 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.322928 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 01 22:32:56.323112 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.323102 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 01 22:32:56.327688 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.327654 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8c776c7f-v7cqr"] Jul 01 22:32:56.329508 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.329487 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 01 22:32:56.477001 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.476956 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-config\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.477123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.477009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-oauth-serving-cert\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.477123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.477077 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxpg7\" (UniqueName: \"kubernetes.io/projected/dee0e561-aced-47f2-9d0c-42ab8d1224f2-kube-api-access-cxpg7\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.477317 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.477242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-serving-cert\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.477317 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.477271 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-oauth-config\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.477317 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.477287 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-service-ca\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.477317 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.477302 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-trusted-ca-bundle\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578366 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.577665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-serving-cert\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578366 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.577714 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-oauth-config\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578366 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.577742 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-service-ca\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578366 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.577767 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-trusted-ca-bundle\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578366 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.577802 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-config\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578366 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.577832 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-oauth-serving-cert\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578366 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.577857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxpg7\" (UniqueName: \"kubernetes.io/projected/dee0e561-aced-47f2-9d0c-42ab8d1224f2-kube-api-access-cxpg7\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578802 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.578663 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-config\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578802 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.578716 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-trusted-ca-bundle\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.578802 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.578763 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-oauth-serving-cert\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.579358 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.579330 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-service-ca\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.581116 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.581088 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-serving-cert\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.581562 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.581437 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-oauth-config\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.593131 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.593084 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxpg7\" (UniqueName: \"kubernetes.io/projected/dee0e561-aced-47f2-9d0c-42ab8d1224f2-kube-api-access-cxpg7\") pod \"console-6b8c776c7f-v7cqr\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.624993 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.624930 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:32:56.797431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:56.797399 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8c776c7f-v7cqr"] Jul 01 22:32:56.801070 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:32:56.801029 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee0e561_aced_47f2_9d0c_42ab8d1224f2.slice/crio-a22043317ef0d4b3ec862a3d9da86e1dad85d4beee0fc26b7335c28dcf3180d3 WatchSource:0}: Error finding container a22043317ef0d4b3ec862a3d9da86e1dad85d4beee0fc26b7335c28dcf3180d3: Status 404 returned error can't find the container with id a22043317ef0d4b3ec862a3d9da86e1dad85d4beee0fc26b7335c28dcf3180d3 Jul 01 22:32:57.017476 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.017403 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4f4kd" event={"ID":"1ae01200-de56-4195-a919-151c0a717eed","Type":"ContainerStarted","Data":"ab5e5e93877585ce3f085661fd9df5474dce557a76585e237b24129fe4f82dd5"} Jul 01 22:32:57.018555 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.018532 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8c776c7f-v7cqr" event={"ID":"dee0e561-aced-47f2-9d0c-42ab8d1224f2","Type":"ContainerStarted","Data":"a22043317ef0d4b3ec862a3d9da86e1dad85d4beee0fc26b7335c28dcf3180d3"} Jul 01 22:32:57.020186 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.020160 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rwvtw" event={"ID":"4eaf10c2-65d3-44bb-a063-2011878aba2b","Type":"ContainerStarted","Data":"3c3b665811f3632f7bd3420a6174013e2053a2b7233180de89577aefb13e7682"} Jul 01 22:32:57.020290 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.020214 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rwvtw" event={"ID":"4eaf10c2-65d3-44bb-a063-2011878aba2b","Type":"ContainerStarted","Data":"509f7023ac5561c012e45905429c788b2e10af5d2a4a873376c106d6ba4ec8b7"} Jul 01 22:32:57.020352 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.020281 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rwvtw" Jul 01 22:32:57.021543 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.021523 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xm7xn" event={"ID":"6aa2539a-b2b3-4e99-a5d2-13c56f919590","Type":"ContainerStarted","Data":"fa79db34753799f8797c54aece7268fd9ce0106ef0f50b386e4a25c79bb3f895"} Jul 01 22:32:57.023093 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.023074 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" event={"ID":"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06","Type":"ContainerStarted","Data":"ad5e6ae9a422a99ac80bdf08d03d3c220fc63b37b61980c6932b8d8e96316fe0"} Jul 01 22:32:57.023184 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.023108 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" event={"ID":"f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06","Type":"ContainerStarted","Data":"901e56bf976d8c2ad8311ceb6959c7e81c22a92647d15f02ca5ee80974fe30d0"} Jul 01 22:32:57.038350 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.038312 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rwvtw" podStartSLOduration=2.301740437 podStartE2EDuration="9.038299261s" totalImagesPullingTime="6.736558824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:57.037846958 +0000 UTC m=+47.925104915" watchObservedRunningTime="2026-07-01 22:32:57.038299261 +0000 UTC m=+47.925557217" Jul 01 22:32:57.056730 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.056684 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xm7xn" podStartSLOduration=2.2998661240000002 podStartE2EDuration="9.056673807s" totalImagesPullingTime="6.756807683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:57.056555232 +0000 UTC m=+47.943813188" watchObservedRunningTime="2026-07-01 22:32:57.056673807 +0000 UTC m=+47.943931764" Jul 01 22:32:57.084171 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:57.084108 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-fkz95" podStartSLOduration=1.420312732 podStartE2EDuration="7.08409301s" totalImagesPullingTime="5.663780278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:57.083356355 +0000 UTC m=+47.970614311" watchObservedRunningTime="2026-07-01 22:32:57.08409301 +0000 UTC m=+47.971350965" Jul 01 22:32:58.028343 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.028303 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4f4kd" event={"ID":"1ae01200-de56-4195-a919-151c0a717eed","Type":"ContainerStarted","Data":"fbf51e5f4e7d453f7c9213ec54836c29870fd688f5d633e7ed04e35ae5932e4a"} Jul 01 22:32:58.566984 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.566719 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2"] Jul 01 22:32:58.595592 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.595555 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2"] Jul 01 22:32:58.595766 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.595612 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-bxhr5"] Jul 01 22:32:58.595766 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.595663 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.599224 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.598692 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hqzrn\"" Jul 01 22:32:58.599224 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.598958 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 01 22:32:58.601876 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.599429 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 01 22:32:58.620382 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.620347 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-bxhr5"] Jul 01 22:32:58.620531 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.620451 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.621021 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.620996 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6rz7c"] Jul 01 22:32:58.623602 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.623555 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 01 22:32:58.625062 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.624876 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 01 22:32:58.625062 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.624945 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 01 22:32:58.625062 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.624982 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-g9zwl\"" Jul 01 22:32:58.635400 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.635378 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.638674 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.638608 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 22:32:58.638674 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.638629 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 22:32:58.639012 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.638903 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zwg2h\"" Jul 01 22:32:58.639130 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.639098 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 22:32:58.698525 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.698477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.698717 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.698562 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h682\" (UniqueName: \"kubernetes.io/projected/f6d438d2-58b4-445e-aac3-b08dc12f116c-kube-api-access-4h682\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.698717 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.698595 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.698717 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.698621 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6d438d2-58b4-445e-aac3-b08dc12f116c-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.799307 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799274 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42677453-af61-4eca-8b32-38c8c59f455a-metrics-client-ca\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.799493 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799317 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gfm5\" (UniqueName: \"kubernetes.io/projected/1595ad19-6cd6-423a-8753-b2346148bf55-kube-api-access-2gfm5\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.799493 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6d438d2-58b4-445e-aac3-b08dc12f116c-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.799493 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799383 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.799493 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799413 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-tls\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.799493 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1595ad19-6cd6-423a-8753-b2346148bf55-metrics-client-ca\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.799493 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799484 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-textfile\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799560 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-accelerators-collector-config\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcjs4\" (UniqueName: \"kubernetes.io/projected/42677453-af61-4eca-8b32-38c8c59f455a-kube-api-access-jcjs4\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799647 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-wtmp\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1595ad19-6cd6-423a-8753-b2346148bf55-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799740 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h682\" (UniqueName: \"kubernetes.io/projected/f6d438d2-58b4-445e-aac3-b08dc12f116c-kube-api-access-4h682\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799765 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-root\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.799800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.799788 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-sys\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.800388 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:58.799944 2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 01 22:32:58.800388 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:58.800009 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-tls podName:f6d438d2-58b4-445e-aac3-b08dc12f116c nodeName:}" failed. No retries permitted until 2026-07-01 22:32:59.299996387 +0000 UTC m=+50.187254321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-tls") pod "openshift-state-metrics-57f99c8b49-l6hc2" (UID: "f6d438d2-58b4-445e-aac3-b08dc12f116c") : secret "openshift-state-metrics-tls" not found Jul 01 22:32:58.800388 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.800166 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6d438d2-58b4-445e-aac3-b08dc12f116c-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.803017 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.802980 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.814093 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.814038 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h682\" (UniqueName: \"kubernetes.io/projected/f6d438d2-58b4-445e-aac3-b08dc12f116c-kube-api-access-4h682\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:58.901113 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-sys\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901113 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42677453-af61-4eca-8b32-38c8c59f455a-metrics-client-ca\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901115 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gfm5\" (UniqueName: \"kubernetes.io/projected/1595ad19-6cd6-423a-8753-b2346148bf55-kube-api-access-2gfm5\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901144 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-sys\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901163 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-tls\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901212 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1595ad19-6cd6-423a-8753-b2346148bf55-metrics-client-ca\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:58.901257 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901265 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901315 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-textfile\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901346 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:58.901335 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-tls podName:42677453-af61-4eca-8b32-38c8c59f455a nodeName:}" failed. No retries permitted until 2026-07-01 22:32:59.401316754 +0000 UTC m=+50.288574705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-tls") pod "node-exporter-6rz7c" (UID: "42677453-af61-4eca-8b32-38c8c59f455a") : secret "node-exporter-tls" not found Jul 01 22:32:58.901765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901361 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-accelerators-collector-config\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901387 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcjs4\" (UniqueName: \"kubernetes.io/projected/42677453-af61-4eca-8b32-38c8c59f455a-kube-api-access-jcjs4\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901411 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.901765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.901765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-wtmp\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901564 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1595ad19-6cd6-423a-8753-b2346148bf55-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.901765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-root\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.901765 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901703 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-root\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.902189 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901842 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-wtmp\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.902189 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901911 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-textfile\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.902189 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.901988 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1595ad19-6cd6-423a-8753-b2346148bf55-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.902346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.902224 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42677453-af61-4eca-8b32-38c8c59f455a-metrics-client-ca\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.902346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.902280 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-accelerators-collector-config\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.902346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.902299 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.902637 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.902576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1595ad19-6cd6-423a-8753-b2346148bf55-metrics-client-ca\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.904569 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.904441 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.904845 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.904822 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.904971 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.904946 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1595ad19-6cd6-423a-8753-b2346148bf55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.913334 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.913292 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcjs4\" (UniqueName: \"kubernetes.io/projected/42677453-af61-4eca-8b32-38c8c59f455a-kube-api-access-jcjs4\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:58.915095 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.915067 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gfm5\" (UniqueName: \"kubernetes.io/projected/1595ad19-6cd6-423a-8753-b2346148bf55-kube-api-access-2gfm5\") pod \"kube-state-metrics-845b954455-bxhr5\" (UID: \"1595ad19-6cd6-423a-8753-b2346148bf55\") " pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:58.931409 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:58.931378 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" Jul 01 22:32:59.305789 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.305753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:59.308513 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.308486 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6d438d2-58b4-445e-aac3-b08dc12f116c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-l6hc2\" (UID: \"f6d438d2-58b4-445e-aac3-b08dc12f116c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:59.406371 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.406327 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-tls\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:59.408954 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.408921 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42677453-af61-4eca-8b32-38c8c59f455a-node-exporter-tls\") pod \"node-exporter-6rz7c\" (UID: \"42677453-af61-4eca-8b32-38c8c59f455a\") " pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:59.510816 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.510773 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" Jul 01 22:32:59.547039 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.547002 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6rz7c" Jul 01 22:32:59.682228 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.682197 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:32:59.704788 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.704765 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.709429 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.709399 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 01 22:32:59.709562 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.709479 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 01 22:32:59.710479 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.709738 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 01 22:32:59.710479 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.709868 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 01 22:32:59.710479 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.709874 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 01 22:32:59.710479 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.709900 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 01 22:32:59.710479 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.710001 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fz6tq\"" Jul 01 22:32:59.710781 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.710487 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 01 22:32:59.713843 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.710968 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 01 22:32:59.713843 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.711210 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 01 22:32:59.726357 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.726336 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:32:59.810415 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810375 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.810593 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810433 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.810593 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810488 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.810702 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-config-volume\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.810702 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.810812 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810771 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-config-out\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.810868 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810832 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.810924 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810903 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.810974 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810949 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-web-config\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.811026 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.810978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.811026 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.811000 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.811152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.811043 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.811152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.811092 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq9sv\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-kube-api-access-nq9sv\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.911978 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.911896 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-web-config\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.911978 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.911936 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.911978 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.911962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912262 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.911999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912262 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq9sv\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-kube-api-access-nq9sv\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912262 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912262 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912185 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912461 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912264 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912461 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-config-volume\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912461 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912332 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912461 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912380 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-config-out\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912461 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912410 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912461 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912448 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.912958 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.912891 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.914422 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.913310 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.914422 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:32:59.913462 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle podName:8a1d366a-b0a6-49f2-933a-64b02259a929 nodeName:}" failed. No retries permitted until 2026-07-01 22:33:00.413442771 +0000 UTC m=+51.300700723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929") : configmap references non-existent config key: ca-bundle.crt Jul 01 22:32:59.915454 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.915145 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.915454 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.915413 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.915454 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.915432 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-config-out\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.915816 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.915790 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.916302 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.916275 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-config-volume\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.916938 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.916907 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.917182 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.917161 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-web-config\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.917247 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.917211 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.917927 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.917906 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:32:59.925068 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:32:59.924985 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq9sv\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-kube-api-access-nq9sv\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:33:00.417834 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.417797 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:33:00.418839 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.418811 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:33:00.616882 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.616843 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:33:00.709788 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.709754 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9"] Jul 01 22:33:00.741188 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.741077 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9"] Jul 01 22:33:00.741188 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.741186 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:00.744458 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.744428 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 01 22:33:00.744612 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.744589 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 01 22:33:00.744761 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.744740 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qkmd5\"" Jul 01 22:33:00.747532 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.747422 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 01 22:33:00.747532 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.747441 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 01 22:33:00.747532 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.747453 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 01 22:33:00.747723 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.747637 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-17kusku0hs4tf\"" Jul 01 22:33:00.922699 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.922660 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-tls\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:00.922699 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.922703 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-grpc-tls\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:00.922907 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.922728 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:00.922907 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.922791 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:00.922907 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.922833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-metrics-client-ca\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:00.922907 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.922865 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k64b\" (UniqueName: \"kubernetes.io/projected/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-kube-api-access-2k64b\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:00.922907 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.922890 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:00.923152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:00.922948 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.024359 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.024278 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-tls\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.024359 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.024318 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-grpc-tls\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.024359 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.024341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.024638 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.024465 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.024638 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.024523 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-metrics-client-ca\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.024638 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.024569 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k64b\" (UniqueName: \"kubernetes.io/projected/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-kube-api-access-2k64b\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.024638 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.024607 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.024638 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.024635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.025422 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.025343 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-metrics-client-ca\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.027145 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.027117 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-tls\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.027361 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.027327 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.027456 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.027397 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.027456 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.027416 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-grpc-tls\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.027544 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.027488 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.027578 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.027566 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.033790 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.033767 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k64b\" (UniqueName: \"kubernetes.io/projected/2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2-kube-api-access-2k64b\") pod \"thanos-querier-5bfd5bf976-qwkx9\" (UID: \"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2\") " pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.053084 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.053042 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:01.471874 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.471830 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-bxhr5"] Jul 01 22:33:01.474940 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:01.474912 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1595ad19_6cd6_423a_8753_b2346148bf55.slice/crio-7d657507d924953465260ad8ae0b87d5e80ef21c8f7fc50c4dcf24e7fdbe888a WatchSource:0}: Error finding container 7d657507d924953465260ad8ae0b87d5e80ef21c8f7fc50c4dcf24e7fdbe888a: Status 404 returned error can't find the container with id 7d657507d924953465260ad8ae0b87d5e80ef21c8f7fc50c4dcf24e7fdbe888a Jul 01 22:33:01.707035 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.707009 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2"] Jul 01 22:33:01.708601 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:01.708573 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d438d2_58b4_445e_aac3_b08dc12f116c.slice/crio-343a8dddfa19f218499bbb3987764adf3be91bb7eae036bd03795579cc56a523 WatchSource:0}: Error finding container 343a8dddfa19f218499bbb3987764adf3be91bb7eae036bd03795579cc56a523: Status 404 returned error can't find the container with id 343a8dddfa19f218499bbb3987764adf3be91bb7eae036bd03795579cc56a523 Jul 01 22:33:01.710737 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.710700 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9"] Jul 01 22:33:01.713707 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:01.713683 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba3edfb_832f_4e7d_bc1a_f1c4406cbfe2.slice/crio-f59fac8c0b79e078c8bdeb1f82a08df43a5daee6d737c898c53f43ad4d46504d WatchSource:0}: Error finding container f59fac8c0b79e078c8bdeb1f82a08df43a5daee6d737c898c53f43ad4d46504d: Status 404 returned error can't find the container with id f59fac8c0b79e078c8bdeb1f82a08df43a5daee6d737c898c53f43ad4d46504d Jul 01 22:33:01.720074 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:01.720036 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:33:01.723150 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:01.723127 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1d366a_b0a6_49f2_933a_64b02259a929.slice/crio-8edab9efea573bdc82503d041b09e3a95a8b09107a04d6a9a57798c264ea5fc6 WatchSource:0}: Error finding container 8edab9efea573bdc82503d041b09e3a95a8b09107a04d6a9a57798c264ea5fc6: Status 404 returned error can't find the container with id 8edab9efea573bdc82503d041b09e3a95a8b09107a04d6a9a57798c264ea5fc6 Jul 01 22:33:02.049642 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.049540 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" event={"ID":"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2","Type":"ContainerStarted","Data":"f59fac8c0b79e078c8bdeb1f82a08df43a5daee6d737c898c53f43ad4d46504d"} Jul 01 22:33:02.051066 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.051021 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerStarted","Data":"8edab9efea573bdc82503d041b09e3a95a8b09107a04d6a9a57798c264ea5fc6"} Jul 01 22:33:02.052380 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.052355 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" event={"ID":"1595ad19-6cd6-423a-8753-b2346148bf55","Type":"ContainerStarted","Data":"7d657507d924953465260ad8ae0b87d5e80ef21c8f7fc50c4dcf24e7fdbe888a"} Jul 01 22:33:02.053871 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.053842 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6rz7c" event={"ID":"42677453-af61-4eca-8b32-38c8c59f455a","Type":"ContainerStarted","Data":"87b7af683b35adcc91fd5c8834e971d564d227d48908d82724e7ee7d5e9d7e92"} Jul 01 22:33:02.056531 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.056504 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4f4kd" event={"ID":"1ae01200-de56-4195-a919-151c0a717eed","Type":"ContainerStarted","Data":"cc6f4bd3f92781dd67d9fabaf0a6f56dd564967dd010bf55b23d1c3aeaa9e4e1"} Jul 01 22:33:02.058309 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.058284 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8c776c7f-v7cqr" event={"ID":"dee0e561-aced-47f2-9d0c-42ab8d1224f2","Type":"ContainerStarted","Data":"eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2"} Jul 01 22:33:02.060242 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.060222 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" event={"ID":"f6d438d2-58b4-445e-aac3-b08dc12f116c","Type":"ContainerStarted","Data":"9f52dbdf3554ab6d0c097c4097ad7b2a8500ebbd67676b13ccee1c2a66a683e9"} Jul 01 22:33:02.060333 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.060261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" event={"ID":"f6d438d2-58b4-445e-aac3-b08dc12f116c","Type":"ContainerStarted","Data":"cfa87c73ea1d4dfb57365370f7461b81491e9bbcebc992b547bcaab4101feecb"} Jul 01 22:33:02.060333 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.060284 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" event={"ID":"f6d438d2-58b4-445e-aac3-b08dc12f116c","Type":"ContainerStarted","Data":"343a8dddfa19f218499bbb3987764adf3be91bb7eae036bd03795579cc56a523"} Jul 01 22:33:02.089186 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.089134 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4f4kd" podStartSLOduration=1.578547355 podStartE2EDuration="13.089117434s" totalImagesPullingTime="11.510570079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:02.088604802 +0000 UTC m=+52.975862760" watchObservedRunningTime="2026-07-01 22:33:02.089117434 +0000 UTC m=+52.976375390" Jul 01 22:33:02.129463 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:02.129409 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b8c776c7f-v7cqr" podStartSLOduration=1.621232293 podStartE2EDuration="6.129391612s" totalImagesPullingTime="4.508159319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:02.127864829 +0000 UTC m=+53.015122789" watchObservedRunningTime="2026-07-01 22:33:02.129391612 +0000 UTC m=+53.016649571" Jul 01 22:33:03.086716 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.085340 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-bd5b6998f-wrwcv"] Jul 01 22:33:03.106160 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.104542 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.108690 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.108662 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ltvfz\"" Jul 01 22:33:03.109530 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.108925 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 01 22:33:03.111506 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.111481 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bd5b6998f-wrwcv"] Jul 01 22:33:03.112859 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.112838 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 01 22:33:03.113964 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.113489 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 22:33:03.113964 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.113750 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 01 22:33:03.114220 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.114108 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1bibi7djf5t1e\"" Jul 01 22:33:03.145157 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.145112 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6be1d7e-16b9-4a73-8149-309a42dbebb9-metrics-server-audit-profiles\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.145353 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.145165 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-secret-metrics-server-client-certs\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.145353 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.145221 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-secret-metrics-server-tls\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.145353 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.145268 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-client-ca-bundle\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.145524 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.145381 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bg4c\" (UniqueName: \"kubernetes.io/projected/f6be1d7e-16b9-4a73-8149-309a42dbebb9-kube-api-access-5bg4c\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.145524 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.145409 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6be1d7e-16b9-4a73-8149-309a42dbebb9-audit-log\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.145524 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.145472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6be1d7e-16b9-4a73-8149-309a42dbebb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.246483 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.246444 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6be1d7e-16b9-4a73-8149-309a42dbebb9-metrics-server-audit-profiles\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.246673 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.246497 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-secret-metrics-server-client-certs\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.246673 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.246534 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-secret-metrics-server-tls\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.246673 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.246561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-client-ca-bundle\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.246673 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.246603 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bg4c\" (UniqueName: \"kubernetes.io/projected/f6be1d7e-16b9-4a73-8149-309a42dbebb9-kube-api-access-5bg4c\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.246673 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.246636 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6be1d7e-16b9-4a73-8149-309a42dbebb9-audit-log\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.246923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.246693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6be1d7e-16b9-4a73-8149-309a42dbebb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.247457 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.247432 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6be1d7e-16b9-4a73-8149-309a42dbebb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.248025 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.247703 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6be1d7e-16b9-4a73-8149-309a42dbebb9-metrics-server-audit-profiles\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.248456 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.248430 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6be1d7e-16b9-4a73-8149-309a42dbebb9-audit-log\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.250834 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.250784 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-client-ca-bundle\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.253672 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.253610 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-secret-metrics-server-tls\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.254356 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.254328 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6be1d7e-16b9-4a73-8149-309a42dbebb9-secret-metrics-server-client-certs\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.257279 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.257231 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bg4c\" (UniqueName: \"kubernetes.io/projected/f6be1d7e-16b9-4a73-8149-309a42dbebb9-kube-api-access-5bg4c\") pod \"metrics-server-bd5b6998f-wrwcv\" (UID: \"f6be1d7e-16b9-4a73-8149-309a42dbebb9\") " pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.366800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.366763 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl"] Jul 01 22:33:03.381653 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.381622 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:03.384955 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.384921 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl"] Jul 01 22:33:03.388628 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.388604 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 01 22:33:03.389428 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.389370 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9vm5t\"" Jul 01 22:33:03.423822 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.423790 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:03.449669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.449592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68psx\" (UniqueName: \"kubernetes.io/projected/7c63d7a1-9f56-48da-8282-f2d3d6894b42-kube-api-access-68psx\") pod \"monitoring-plugin-cd8656b8-fw6cl\" (UID: \"7c63d7a1-9f56-48da-8282-f2d3d6894b42\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:03.449669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.449647 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c63d7a1-9f56-48da-8282-f2d3d6894b42-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-fw6cl\" (UID: \"7c63d7a1-9f56-48da-8282-f2d3d6894b42\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:03.550980 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.550497 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68psx\" (UniqueName: \"kubernetes.io/projected/7c63d7a1-9f56-48da-8282-f2d3d6894b42-kube-api-access-68psx\") pod \"monitoring-plugin-cd8656b8-fw6cl\" (UID: \"7c63d7a1-9f56-48da-8282-f2d3d6894b42\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:03.550980 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.550551 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c63d7a1-9f56-48da-8282-f2d3d6894b42-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-fw6cl\" (UID: \"7c63d7a1-9f56-48da-8282-f2d3d6894b42\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:03.554201 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.554168 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c63d7a1-9f56-48da-8282-f2d3d6894b42-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-fw6cl\" (UID: \"7c63d7a1-9f56-48da-8282-f2d3d6894b42\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:03.559635 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.559606 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68psx\" (UniqueName: \"kubernetes.io/projected/7c63d7a1-9f56-48da-8282-f2d3d6894b42-kube-api-access-68psx\") pod \"monitoring-plugin-cd8656b8-fw6cl\" (UID: \"7c63d7a1-9f56-48da-8282-f2d3d6894b42\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:03.585528 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.585496 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bd5b6998f-wrwcv"] Jul 01 22:33:03.716887 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:03.716856 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:03.885233 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:03.885154 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6be1d7e_16b9_4a73_8149_309a42dbebb9.slice/crio-f2a1f7c79d785a5be5e34919e41cc8489dd5cf675d2e27eb174dbdbf18c70946 WatchSource:0}: Error finding container f2a1f7c79d785a5be5e34919e41cc8489dd5cf675d2e27eb174dbdbf18c70946: Status 404 returned error can't find the container with id f2a1f7c79d785a5be5e34919e41cc8489dd5cf675d2e27eb174dbdbf18c70946 Jul 01 22:33:04.070132 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.070087 2581 generic.go:356] "Generic (PLEG): container finished" podID="42677453-af61-4eca-8b32-38c8c59f455a" containerID="89b170f5d4a679677831ffb228f238ce0f1ae4d5cda389d1861a60010b2400b3" exitCode=0 Jul 01 22:33:04.070359 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.070186 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6rz7c" event={"ID":"42677453-af61-4eca-8b32-38c8c59f455a","Type":"ContainerDied","Data":"89b170f5d4a679677831ffb228f238ce0f1ae4d5cda389d1861a60010b2400b3"} Jul 01 22:33:04.071552 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.071528 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" event={"ID":"f6be1d7e-16b9-4a73-8149-309a42dbebb9","Type":"ContainerStarted","Data":"f2a1f7c79d785a5be5e34919e41cc8489dd5cf675d2e27eb174dbdbf18c70946"} Jul 01 22:33:04.945223 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.945175 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:33:04.967680 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.967639 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:04.969295 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.969270 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:33:04.970667 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.970630 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 22:33:04.970667 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.970647 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 22:33:04.970849 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.970648 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wvbh2\"" Jul 01 22:33:04.970849 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.970639 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 22:33:04.971009 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.970988 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 22:33:04.971009 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.971019 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 22:33:04.971197 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.971062 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 22:33:04.971481 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.971464 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 22:33:04.972656 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.972617 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 22:33:04.973064 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.972958 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dtuc3vhh28gal\"" Jul 01 22:33:04.973307 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.973272 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 22:33:04.973397 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.973292 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 22:33:04.973505 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.973484 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 22:33:04.973561 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.973531 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 22:33:04.977410 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.977261 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 22:33:04.980177 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:04.980154 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 22:33:05.066002 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.065963 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066002 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066005 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066236 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066029 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066236 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066236 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066130 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066236 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066174 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066236 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066219 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066430 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066430 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066313 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066430 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066339 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066430 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066360 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066430 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066380 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngx49\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-kube-api-access-ngx49\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066430 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066425 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-config-out\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066449 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066490 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066515 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-web-config\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066584 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.066669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.066668 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-config\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168096 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168029 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-web-config\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168282 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168145 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168282 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168184 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168282 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-config\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168444 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168293 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168498 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168557 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168521 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168609 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168551 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168609 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168583 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168702 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168613 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168702 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168702 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168684 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168899 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168722 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168899 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168899 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168782 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168899 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngx49\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-kube-api-access-ngx49\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168899 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168828 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.168899 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.168882 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-config-out\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.169439 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.169361 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.169816 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.169549 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.169816 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.169635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.169816 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.169678 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.169816 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.169711 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.170206 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.170183 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.170312 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.170228 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.173918 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.171509 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-config\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.173918 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.171567 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.173918 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.171924 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.173918 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.172534 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.173918 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.172927 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-web-config\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.173918 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.173406 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.173918 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.173844 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.175494 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.175446 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.175583 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.175531 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.175654 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.175589 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.175710 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.175676 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.175874 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.175852 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.176588 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.176571 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.176722 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.176701 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-config-out\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.184073 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.184027 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngx49\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-kube-api-access-ngx49\") pod \"prometheus-k8s-0\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.281357 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.281322 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:05.514063 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.513918 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8c776c7f-v7cqr"] Jul 01 22:33:05.545619 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.545549 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl"] Jul 01 22:33:05.591421 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:05.591148 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:33:05.594434 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:05.594391 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69842bd9_bda4_4155_a486_cf50ff052a2e.slice/crio-661d362c1b8bfd56e1e6cbde8f84d3b9656b49c1408a1afc460fbda07b676888 WatchSource:0}: Error finding container 661d362c1b8bfd56e1e6cbde8f84d3b9656b49c1408a1afc460fbda07b676888: Status 404 returned error can't find the container with id 661d362c1b8bfd56e1e6cbde8f84d3b9656b49c1408a1afc460fbda07b676888 Jul 01 22:33:06.082422 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.082348 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" event={"ID":"7c63d7a1-9f56-48da-8282-f2d3d6894b42","Type":"ContainerStarted","Data":"9f387a9edf0c184346cf6e1feda7620fbc017f42bd4a422eb22bba79f7c8ec93"} Jul 01 22:33:06.084303 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.084281 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" event={"ID":"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2","Type":"ContainerStarted","Data":"3c2fb4debb25a5adc8119f28de1725fcfccf49facc427948485f8ab9cafa354b"} Jul 01 22:33:06.084405 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.084336 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" event={"ID":"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2","Type":"ContainerStarted","Data":"41bc6338eae46f85969b729d561eed1e05490b17998724465b6d96efb4fb1e50"} Jul 01 22:33:06.084405 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.084376 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" event={"ID":"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2","Type":"ContainerStarted","Data":"08240c2436247166227ef7eed6a5cf632814b003a43b2e0e5d8d26d7aa19d145"} Jul 01 22:33:06.085617 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.085595 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerID="ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e" exitCode=0 Jul 01 22:33:06.085713 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.085665 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerDied","Data":"ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e"} Jul 01 22:33:06.087823 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.087798 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" event={"ID":"1595ad19-6cd6-423a-8753-b2346148bf55","Type":"ContainerStarted","Data":"60b0edcbc128ca8477d2839cc686170995fbd8480aec251540ea8d44dfd91d2f"} Jul 01 22:33:06.087905 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.087865 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" event={"ID":"1595ad19-6cd6-423a-8753-b2346148bf55","Type":"ContainerStarted","Data":"0dc26faf4127b9e61b9f36bb2d93a46426f84535123cf0d3f4b3160c0cbab6c7"} Jul 01 22:33:06.087905 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.087890 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" event={"ID":"1595ad19-6cd6-423a-8753-b2346148bf55","Type":"ContainerStarted","Data":"edcd0489efab434dd697cc3946e553b6dc27b93009bb21b85d2f175405a416f4"} Jul 01 22:33:06.089909 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.089888 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6rz7c" event={"ID":"42677453-af61-4eca-8b32-38c8c59f455a","Type":"ContainerStarted","Data":"5253f3dfb7c3098560cec8b0c12e0a15f81539053ef10797d47fd1bde2fa0b7c"} Jul 01 22:33:06.090003 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.089931 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6rz7c" event={"ID":"42677453-af61-4eca-8b32-38c8c59f455a","Type":"ContainerStarted","Data":"9a7ea584721e6b434348bf05083266f43198cd650a54800d08d91c15d3d68231"} Jul 01 22:33:06.091444 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.091423 2581 generic.go:356] "Generic (PLEG): container finished" podID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" exitCode=0 Jul 01 22:33:06.091548 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.091505 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerDied","Data":"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961"} Jul 01 22:33:06.091615 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.091583 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerStarted","Data":"661d362c1b8bfd56e1e6cbde8f84d3b9656b49c1408a1afc460fbda07b676888"} Jul 01 22:33:06.093883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.093863 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" event={"ID":"f6d438d2-58b4-445e-aac3-b08dc12f116c","Type":"ContainerStarted","Data":"d962e71cf7e7300f4351a7ab12aadb17da1e3784606c56a66f00575dbaac8af2"} Jul 01 22:33:06.172994 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.172939 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-l6hc2" podStartSLOduration=4.7424549240000005 podStartE2EDuration="8.172920565s" totalImagesPullingTime="3.430465641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:06.171558122 +0000 UTC m=+57.058816107" watchObservedRunningTime="2026-07-01 22:33:06.172920565 +0000 UTC m=+57.060178522" Jul 01 22:33:06.191547 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.191499 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-bxhr5" podStartSLOduration=4.541637776 podStartE2EDuration="8.191482456s" totalImagesPullingTime="3.64984468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:06.190711616 +0000 UTC m=+57.077969577" watchObservedRunningTime="2026-07-01 22:33:06.191482456 +0000 UTC m=+57.078740424" Jul 01 22:33:06.212003 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.211812 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6rz7c" podStartSLOduration=6.079649873 podStartE2EDuration="8.211793237s" totalImagesPullingTime="2.033261351s" totalInitContainerRuntime="98.882013ms" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:06.211204544 +0000 UTC m=+57.098462547" watchObservedRunningTime="2026-07-01 22:33:06.211793237 +0000 UTC m=+57.099051194" Jul 01 22:33:06.626108 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:06.626072 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:33:07.034180 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:07.033891 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rwvtw" Jul 01 22:33:08.120492 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:08.120395 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" event={"ID":"f6be1d7e-16b9-4a73-8149-309a42dbebb9","Type":"ContainerStarted","Data":"4a801136372d9a00fd33246cbb1926be4d3f71bf7dcc2f6320d94de7800b2c81"} Jul 01 22:33:08.143009 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:08.142942 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" podStartSLOduration=1.571895482 podStartE2EDuration="5.142922695s" totalImagesPullingTime="3.571027213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:08.14151909 +0000 UTC m=+59.028777061" watchObservedRunningTime="2026-07-01 22:33:08.142922695 +0000 UTC m=+59.030180653" Jul 01 22:33:09.132115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.131911 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" event={"ID":"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2","Type":"ContainerStarted","Data":"5de812fdaa22ef8325300117b4166ecbbb0a817eef7fb2c15fb5b3025afc4272"} Jul 01 22:33:09.132115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.132036 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" event={"ID":"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2","Type":"ContainerStarted","Data":"647c8084356dc2bcc563dd64cb9bcd83c3a284d1153abddac6569990fceac70f"} Jul 01 22:33:09.132115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.132105 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" event={"ID":"2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2","Type":"ContainerStarted","Data":"3c7c5c188b246774f282795cb8653361235e1f2bf5779c1bed48261a6ef1e8f5"} Jul 01 22:33:09.132800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.132217 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:09.135661 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.135627 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerStarted","Data":"1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a"} Jul 01 22:33:09.135825 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.135704 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerStarted","Data":"ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4"} Jul 01 22:33:09.135825 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.135738 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerStarted","Data":"8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac"} Jul 01 22:33:09.137439 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.137409 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" event={"ID":"7c63d7a1-9f56-48da-8282-f2d3d6894b42","Type":"ContainerStarted","Data":"bb6f4ca878c214395cd83d813289f552091d33788a8986e8340c59be8530bada"} Jul 01 22:33:09.138130 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.137837 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:09.143650 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.143631 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" Jul 01 22:33:09.161303 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.161242 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" podStartSLOduration=2.725249683 podStartE2EDuration="9.161224394s" totalImagesPullingTime="6.435974711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:33:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:09.159203471 +0000 UTC m=+60.046461445" watchObservedRunningTime="2026-07-01 22:33:09.161224394 +0000 UTC m=+60.048482364" Jul 01 22:33:09.180667 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:09.180600 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-fw6cl" podStartSLOduration=3.05852131 podStartE2EDuration="6.180577531s" totalImagesPullingTime="3.122056221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:09.179940858 +0000 UTC m=+60.067198816" watchObservedRunningTime="2026-07-01 22:33:09.180577531 +0000 UTC m=+60.067835489" Jul 01 22:33:10.155168 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:10.155137 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5bfd5bf976-qwkx9" Jul 01 22:33:10.958068 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:10.958029 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95fqr" Jul 01 22:33:11.160861 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.160817 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerStarted","Data":"88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529"} Jul 01 22:33:11.161338 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.160949 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerStarted","Data":"dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2"} Jul 01 22:33:11.161338 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.160999 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerStarted","Data":"5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd"} Jul 01 22:33:11.164783 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.164752 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerStarted","Data":"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f"} Jul 01 22:33:11.164959 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.164937 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerStarted","Data":"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb"} Jul 01 22:33:11.165038 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.164991 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerStarted","Data":"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df"} Jul 01 22:33:11.165113 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.165038 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerStarted","Data":"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c"} Jul 01 22:33:11.165113 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.165105 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerStarted","Data":"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f"} Jul 01 22:33:11.165225 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.165148 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerStarted","Data":"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0"} Jul 01 22:33:11.329841 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.329737 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.084114086 podStartE2EDuration="12.329715126s" totalImagesPullingTime="6.24560104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:11.231903503 +0000 UTC m=+62.119161461" watchObservedRunningTime="2026-07-01 22:33:11.329715126 +0000 UTC m=+62.216973130" Jul 01 22:33:11.330005 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:11.329984 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.194574728 podStartE2EDuration="7.329977903s" totalImagesPullingTime="4.102727469s" totalInitContainerRuntime="32.675706ms" isStatefulPod=false podCreationTimestamp="2026-07-01 22:33:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:11.329390909 +0000 UTC m=+62.216648903" watchObservedRunningTime="2026-07-01 22:33:11.329977903 +0000 UTC m=+62.217235860" Jul 01 22:33:14.272327 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.272288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:33:14.275467 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.275444 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 22:33:14.284897 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.284879 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a25f676f-13ae-48b5-b613-04af7f4f078e-metrics-certs\") pod \"network-metrics-daemon-2dxrj\" (UID: \"a25f676f-13ae-48b5-b613-04af7f4f078e\") " pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:33:14.373423 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.373385 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:33:14.376502 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.376479 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 22:33:14.385992 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.385962 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/889ac5e9-344c-4000-8257-66a0ef85e0eb-original-pull-secret\") pod \"global-pull-secret-syncer-4g9k9\" (UID: \"889ac5e9-344c-4000-8257-66a0ef85e0eb\") " pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:33:14.397812 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.397793 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v2q69\"" Jul 01 22:33:14.405276 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.405259 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dxrj" Jul 01 22:33:14.474236 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.474205 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:33:14.478897 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.478874 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 22:33:14.488188 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.488162 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 22:33:14.498226 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.498196 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4df5\" (UniqueName: \"kubernetes.io/projected/fd36e128-b67d-4e16-acdf-b755b5fdbbca-kube-api-access-c4df5\") pod \"network-check-target-gwf6l\" (UID: \"fd36e128-b67d-4e16-acdf-b755b5fdbbca\") " pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:33:14.532295 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.532273 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2dxrj"] Jul 01 22:33:14.534609 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:14.534582 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25f676f_13ae_48b5_b613_04af7f4f078e.slice/crio-9a5dc010e90dcd0558d693172f9c287a53d7dff05b278d696939cc1b2e084c42 WatchSource:0}: Error finding container 9a5dc010e90dcd0558d693172f9c287a53d7dff05b278d696939cc1b2e084c42: Status 404 returned error can't find the container with id 9a5dc010e90dcd0558d693172f9c287a53d7dff05b278d696939cc1b2e084c42 Jul 01 22:33:14.677771 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.677735 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7csn6\"" Jul 01 22:33:14.685155 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.685124 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:33:14.686412 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.686391 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4g9k9" Jul 01 22:33:14.816819 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.816791 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4g9k9"] Jul 01 22:33:14.819247 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:14.819224 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889ac5e9_344c_4000_8257_66a0ef85e0eb.slice/crio-8a4c35fa1e1df4e6ad250be4deea98d24b158a8c5fe33900ee53f545bd7fa70b WatchSource:0}: Error finding container 8a4c35fa1e1df4e6ad250be4deea98d24b158a8c5fe33900ee53f545bd7fa70b: Status 404 returned error can't find the container with id 8a4c35fa1e1df4e6ad250be4deea98d24b158a8c5fe33900ee53f545bd7fa70b Jul 01 22:33:14.835392 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:14.835359 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gwf6l"] Jul 01 22:33:14.838892 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:33:14.838861 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd36e128_b67d_4e16_acdf_b755b5fdbbca.slice/crio-a398ee902d322740247376b9fb002e35b45f40c080c273f4e9b18a980840ed3a WatchSource:0}: Error finding container a398ee902d322740247376b9fb002e35b45f40c080c273f4e9b18a980840ed3a: Status 404 returned error can't find the container with id a398ee902d322740247376b9fb002e35b45f40c080c273f4e9b18a980840ed3a Jul 01 22:33:15.184531 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:15.184441 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gwf6l" event={"ID":"fd36e128-b67d-4e16-acdf-b755b5fdbbca","Type":"ContainerStarted","Data":"a398ee902d322740247376b9fb002e35b45f40c080c273f4e9b18a980840ed3a"} Jul 01 22:33:15.185764 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:15.185712 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4g9k9" event={"ID":"889ac5e9-344c-4000-8257-66a0ef85e0eb","Type":"ContainerStarted","Data":"8a4c35fa1e1df4e6ad250be4deea98d24b158a8c5fe33900ee53f545bd7fa70b"} Jul 01 22:33:15.187101 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:15.187072 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2dxrj" event={"ID":"a25f676f-13ae-48b5-b613-04af7f4f078e","Type":"ContainerStarted","Data":"9a5dc010e90dcd0558d693172f9c287a53d7dff05b278d696939cc1b2e084c42"} Jul 01 22:33:15.281488 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:15.281452 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:16.198296 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:16.196540 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2dxrj" event={"ID":"a25f676f-13ae-48b5-b613-04af7f4f078e","Type":"ContainerStarted","Data":"17af8ed6ace7aa5f8c3a0cf3f7cc4b84bf97c4733966dfec235713eb8be8a859"} Jul 01 22:33:16.198296 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:16.196662 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2dxrj" event={"ID":"a25f676f-13ae-48b5-b613-04af7f4f078e","Type":"ContainerStarted","Data":"a039c182c825956e41772747bf78c3fdd34ad8ec1cacdf12e8464c6b72eeddd1"} Jul 01 22:33:16.214657 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:16.214595 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2dxrj" podStartSLOduration=65.242923864 podStartE2EDuration="1m6.214573751s" totalImagesPullingTime="971.649887ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:16.213079973 +0000 UTC m=+67.100337931" watchObservedRunningTime="2026-07-01 22:33:16.214573751 +0000 UTC m=+67.101831709" Jul 01 22:33:20.231333 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:20.231229 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gwf6l" event={"ID":"fd36e128-b67d-4e16-acdf-b755b5fdbbca","Type":"ContainerStarted","Data":"fd2eca565806e85644874f62fb6b109169575072f69d75a8da0440615bfac368"} Jul 01 22:33:20.231333 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:20.231339 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:33:20.233129 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:20.233091 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4g9k9" event={"ID":"889ac5e9-344c-4000-8257-66a0ef85e0eb","Type":"ContainerStarted","Data":"eaab95e2642b8786c34ae0e3425a6e48da8a48a36d98e65af055bf9b5686abfd"} Jul 01 22:33:20.253586 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:20.253535 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gwf6l" podStartSLOduration=65.137661523 podStartE2EDuration="1m10.253519217s" totalImagesPullingTime="5.115857694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:20.252600659 +0000 UTC m=+71.139858616" watchObservedRunningTime="2026-07-01 22:33:20.253519217 +0000 UTC m=+71.140777152" Jul 01 22:33:20.276129 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:20.276061 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4g9k9" podStartSLOduration=64.126746864 podStartE2EDuration="1m9.276030116s" totalImagesPullingTime="5.149283252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:20.273824935 +0000 UTC m=+71.161082892" watchObservedRunningTime="2026-07-01 22:33:20.276030116 +0000 UTC m=+71.163288077" Jul 01 22:33:23.424479 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:23.424452 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:23.424919 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:23.424537 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:30.556716 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:30.556643 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b8c776c7f-v7cqr" podUID="dee0e561-aced-47f2-9d0c-42ab8d1224f2" containerName="console" containerID="cri-o://eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2" gracePeriod=15 Jul 01 22:33:30.846344 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:30.846308 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:33:31.027889 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.027850 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-service-ca\") pod \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " Jul 01 22:33:31.027889 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.027896 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-oauth-serving-cert\") pod \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " Jul 01 22:33:31.028158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.027919 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-oauth-config\") pod \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " Jul 01 22:33:31.028158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.027940 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-config\") pod \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " Jul 01 22:33:31.028158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.027978 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-trusted-ca-bundle\") pod \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " Jul 01 22:33:31.028158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.028036 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxpg7\" (UniqueName: \"kubernetes.io/projected/dee0e561-aced-47f2-9d0c-42ab8d1224f2-kube-api-access-cxpg7\") pod \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " Jul 01 22:33:31.028158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.028094 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-serving-cert\") pod \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\" (UID: \"dee0e561-aced-47f2-9d0c-42ab8d1224f2\") " Jul 01 22:33:31.028446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.028350 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dee0e561-aced-47f2-9d0c-42ab8d1224f2" (UID: "dee0e561-aced-47f2-9d0c-42ab8d1224f2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:33:31.028446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.028406 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-config" (OuterVolumeSpecName: "console-config") pod "dee0e561-aced-47f2-9d0c-42ab8d1224f2" (UID: "dee0e561-aced-47f2-9d0c-42ab8d1224f2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:33:31.028629 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.028595 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dee0e561-aced-47f2-9d0c-42ab8d1224f2" (UID: "dee0e561-aced-47f2-9d0c-42ab8d1224f2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:33:31.028688 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.028658 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-service-ca" (OuterVolumeSpecName: "service-ca") pod "dee0e561-aced-47f2-9d0c-42ab8d1224f2" (UID: "dee0e561-aced-47f2-9d0c-42ab8d1224f2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:33:31.030533 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.030496 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dee0e561-aced-47f2-9d0c-42ab8d1224f2" (UID: "dee0e561-aced-47f2-9d0c-42ab8d1224f2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:33:31.030648 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.030572 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dee0e561-aced-47f2-9d0c-42ab8d1224f2" (UID: "dee0e561-aced-47f2-9d0c-42ab8d1224f2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:33:31.030648 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.030604 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dee0e561-aced-47f2-9d0c-42ab8d1224f2-kube-api-access-cxpg7" (OuterVolumeSpecName: "kube-api-access-cxpg7") pod "dee0e561-aced-47f2-9d0c-42ab8d1224f2" (UID: "dee0e561-aced-47f2-9d0c-42ab8d1224f2"). InnerVolumeSpecName "kube-api-access-cxpg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:33:31.129239 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.129134 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-trusted-ca-bundle\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:33:31.129239 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.129173 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxpg7\" (UniqueName: \"kubernetes.io/projected/dee0e561-aced-47f2-9d0c-42ab8d1224f2-kube-api-access-cxpg7\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:33:31.129239 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.129187 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-serving-cert\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:33:31.129239 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.129199 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-service-ca\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:33:31.129239 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.129212 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-oauth-serving-cert\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:33:31.129239 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.129224 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-oauth-config\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:33:31.129239 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.129240 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dee0e561-aced-47f2-9d0c-42ab8d1224f2-console-config\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:33:31.289347 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.289223 2581 generic.go:356] "Generic (PLEG): container finished" podID="dee0e561-aced-47f2-9d0c-42ab8d1224f2" containerID="eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2" exitCode=0 Jul 01 22:33:31.289347 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.289289 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8c776c7f-v7cqr" Jul 01 22:33:31.289347 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.289300 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8c776c7f-v7cqr" event={"ID":"dee0e561-aced-47f2-9d0c-42ab8d1224f2","Type":"ContainerDied","Data":"eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2"} Jul 01 22:33:31.289559 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.289372 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8c776c7f-v7cqr" event={"ID":"dee0e561-aced-47f2-9d0c-42ab8d1224f2","Type":"ContainerDied","Data":"a22043317ef0d4b3ec862a3d9da86e1dad85d4beee0fc26b7335c28dcf3180d3"} Jul 01 22:33:31.298529 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.298502 2581 scope.go:117] "RemoveContainer" containerID="eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2" Jul 01 22:33:31.306906 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.306883 2581 scope.go:117] "RemoveContainer" containerID="eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2" Jul 01 22:33:31.307225 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:33:31.307205 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2\": container with ID starting with eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2 not found: ID does not exist" containerID="eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2" Jul 01 22:33:31.307317 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.307235 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2"} err="failed to get container status \"eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2\": rpc error: code = NotFound desc = could not find container \"eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2\": container with ID starting with eabbfd833b79ce5675ecca348806e1bedd14390e2ae8ead2996d6d32391bc7f2 not found: ID does not exist" Jul 01 22:33:31.311955 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.311928 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8c776c7f-v7cqr"] Jul 01 22:33:31.314063 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.314031 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b8c776c7f-v7cqr"] Jul 01 22:33:31.663846 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:31.663812 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dee0e561-aced-47f2-9d0c-42ab8d1224f2" path="/var/lib/kubelet/pods/dee0e561-aced-47f2-9d0c-42ab8d1224f2/volumes" Jul 01 22:33:43.437106 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:43.435457 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:43.441167 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:43.441132 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-bd5b6998f-wrwcv" Jul 01 22:33:51.240783 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:51.240743 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gwf6l" Jul 01 22:33:59.131397 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:59.131352 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:59.151016 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:59.150990 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:33:59.422004 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:33:59.421913 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:09.015371 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.015331 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:34:09.015830 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.015736 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="alertmanager" containerID="cri-o://8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac" gracePeriod=120 Jul 01 22:34:09.015912 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.015819 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy-metric" containerID="cri-o://dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2" gracePeriod=120 Jul 01 22:34:09.015912 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.015871 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy" containerID="cri-o://5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd" gracePeriod=120 Jul 01 22:34:09.016014 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.015893 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="config-reloader" containerID="cri-o://ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4" gracePeriod=120 Jul 01 22:34:09.016014 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.015871 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="prom-label-proxy" containerID="cri-o://88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529" gracePeriod=120 Jul 01 22:34:09.016148 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.015828 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy-web" containerID="cri-o://1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a" gracePeriod=120 Jul 01 22:34:09.449695 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449607 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerID="88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529" exitCode=0 Jul 01 22:34:09.449695 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449634 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerID="dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2" exitCode=0 Jul 01 22:34:09.449695 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449640 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerID="5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd" exitCode=0 Jul 01 22:34:09.449695 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449646 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerID="ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4" exitCode=0 Jul 01 22:34:09.449695 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449651 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerID="8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac" exitCode=0 Jul 01 22:34:09.449695 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449674 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerDied","Data":"88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529"} Jul 01 22:34:09.449973 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449737 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerDied","Data":"dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2"} Jul 01 22:34:09.449973 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449765 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerDied","Data":"5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd"} Jul 01 22:34:09.449973 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449790 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerDied","Data":"ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4"} Jul 01 22:34:09.449973 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:09.449812 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerDied","Data":"8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac"} Jul 01 22:34:10.270591 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.270567 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370619 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-cluster-tls-config\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370667 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-web\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370696 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-tls-assets\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370778 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370819 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-main-db\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370851 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370892 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq9sv\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-kube-api-access-nq9sv\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370938 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-metrics-client-ca\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.370969 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.371012 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-main-tls\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.371040 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-config-out\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.371083 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-web-config\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.371511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.371113 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-config-volume\") pod \"8a1d366a-b0a6-49f2-933a-64b02259a929\" (UID: \"8a1d366a-b0a6-49f2-933a-64b02259a929\") " Jul 01 22:34:10.374109 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.373814 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:10.378835 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.378347 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:34:10.381343 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.381300 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:10.390235 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.388838 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-config-volume" (OuterVolumeSpecName: "config-volume") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:10.396428 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.396395 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:10.396428 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.396407 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-config-out" (OuterVolumeSpecName: "config-out") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:34:10.396428 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.396421 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-kube-api-access-nq9sv" (OuterVolumeSpecName: "kube-api-access-nq9sv") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "kube-api-access-nq9sv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:34:10.396967 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.396931 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:34:10.397100 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.396988 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:10.397244 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.397176 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:10.397324 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.397263 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:10.398815 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.398788 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:10.405458 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.405434 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-web-config" (OuterVolumeSpecName: "web-config") pod "8a1d366a-b0a6-49f2-933a-64b02259a929" (UID: "8a1d366a-b0a6-49f2-933a-64b02259a929"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:10.457662 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.457631 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerID="1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a" exitCode=0 Jul 01 22:34:10.457804 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.457705 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerDied","Data":"1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a"} Jul 01 22:34:10.457804 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.457724 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.457804 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.457802 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8a1d366a-b0a6-49f2-933a-64b02259a929","Type":"ContainerDied","Data":"8edab9efea573bdc82503d041b09e3a95a8b09107a04d6a9a57798c264ea5fc6"} Jul 01 22:34:10.457930 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.457847 2581 scope.go:117] "RemoveContainer" containerID="88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529" Jul 01 22:34:10.466028 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.466009 2581 scope.go:117] "RemoveContainer" containerID="dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2" Jul 01 22:34:10.471799 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471775 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.471887 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471804 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-alertmanager-main-db\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.471887 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471821 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.471887 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471834 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq9sv\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-kube-api-access-nq9sv\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.471887 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471847 2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d366a-b0a6-49f2-933a-64b02259a929-metrics-client-ca\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.471887 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471861 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.471887 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471875 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-main-tls\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.472106 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471890 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a1d366a-b0a6-49f2-933a-64b02259a929-config-out\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.472106 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471903 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-web-config\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.472106 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471915 2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-config-volume\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.472106 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471928 2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-cluster-tls-config\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.472106 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471940 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a1d366a-b0a6-49f2-933a-64b02259a929-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.472106 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.471954 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a1d366a-b0a6-49f2-933a-64b02259a929-tls-assets\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:10.472794 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.472777 2581 scope.go:117] "RemoveContainer" containerID="5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd" Jul 01 22:34:10.479384 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.479368 2581 scope.go:117] "RemoveContainer" containerID="1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a" Jul 01 22:34:10.487036 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.487020 2581 scope.go:117] "RemoveContainer" containerID="ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4" Jul 01 22:34:10.487793 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.487767 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:34:10.493166 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.493143 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:34:10.495871 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.495851 2581 scope.go:117] "RemoveContainer" containerID="8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac" Jul 01 22:34:10.502994 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.502974 2581 scope.go:117] "RemoveContainer" containerID="ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e" Jul 01 22:34:10.510941 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.510917 2581 scope.go:117] "RemoveContainer" containerID="88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529" Jul 01 22:34:10.511229 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:10.511211 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529\": container with ID starting with 88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529 not found: ID does not exist" containerID="88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529" Jul 01 22:34:10.511296 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.511240 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529"} err="failed to get container status \"88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529\": rpc error: code = NotFound desc = could not find container \"88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529\": container with ID starting with 88754965dc69bcefa8a4979eb87762de99e9b767c525f11edf68c6aa5c7b6529 not found: ID does not exist" Jul 01 22:34:10.511296 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.511260 2581 scope.go:117] "RemoveContainer" containerID="dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2" Jul 01 22:34:10.511520 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:10.511502 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2\": container with ID starting with dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2 not found: ID does not exist" containerID="dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2" Jul 01 22:34:10.511588 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.511538 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2"} err="failed to get container status \"dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2\": rpc error: code = NotFound desc = could not find container \"dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2\": container with ID starting with dabc1ca3fe3ec385f19d4d0237e2696f466536fee97c9a98032f47f4d2915ef2 not found: ID does not exist" Jul 01 22:34:10.511588 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.511562 2581 scope.go:117] "RemoveContainer" containerID="5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd" Jul 01 22:34:10.511804 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:10.511787 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd\": container with ID starting with 5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd not found: ID does not exist" containerID="5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd" Jul 01 22:34:10.511852 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.511809 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd"} err="failed to get container status \"5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd\": rpc error: code = NotFound desc = could not find container \"5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd\": container with ID starting with 5dee6009b4bb76d5ee06f2be37bc26dd7975fa553d331a83ce60c45e19ced1fd not found: ID does not exist" Jul 01 22:34:10.511852 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.511824 2581 scope.go:117] "RemoveContainer" containerID="1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a" Jul 01 22:34:10.512090 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:10.512073 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a\": container with ID starting with 1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a not found: ID does not exist" containerID="1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a" Jul 01 22:34:10.512141 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.512095 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a"} err="failed to get container status \"1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a\": rpc error: code = NotFound desc = could not find container \"1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a\": container with ID starting with 1696dfac80c13c2610864704b269ae3a701c9f0959b1f5237da7809a960f774a not found: ID does not exist" Jul 01 22:34:10.512141 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.512109 2581 scope.go:117] "RemoveContainer" containerID="ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4" Jul 01 22:34:10.512320 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:10.512303 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4\": container with ID starting with ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4 not found: ID does not exist" containerID="ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4" Jul 01 22:34:10.512382 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.512325 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4"} err="failed to get container status \"ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4\": rpc error: code = NotFound desc = could not find container \"ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4\": container with ID starting with ac68ead46dd3b2aebb5865afb413860559657e68cea2732866dd27498212d3b4 not found: ID does not exist" Jul 01 22:34:10.512382 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.512338 2581 scope.go:117] "RemoveContainer" containerID="8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac" Jul 01 22:34:10.512534 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:10.512519 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac\": container with ID starting with 8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac not found: ID does not exist" containerID="8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac" Jul 01 22:34:10.512572 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.512537 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac"} err="failed to get container status \"8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac\": rpc error: code = NotFound desc = could not find container \"8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac\": container with ID starting with 8d90cf978eb2536c70a95889cf934b13aec2861adab961001cf4756b1b1f78ac not found: ID does not exist" Jul 01 22:34:10.512572 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.512549 2581 scope.go:117] "RemoveContainer" containerID="ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e" Jul 01 22:34:10.512791 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:10.512773 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e\": container with ID starting with ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e not found: ID does not exist" containerID="ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e" Jul 01 22:34:10.512843 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.512797 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e"} err="failed to get container status \"ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e\": rpc error: code = NotFound desc = could not find container \"ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e\": container with ID starting with ca72c6d8eca68ed5cba948eea1c7affecaf40997945b7440b15a12bb1e8b918e not found: ID does not exist" Jul 01 22:34:10.537893 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.537862 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:34:10.538200 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538187 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy-web" Jul 01 22:34:10.538248 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538202 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy-web" Jul 01 22:34:10.538248 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538216 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="init-config-reloader" Jul 01 22:34:10.538248 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538222 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="init-config-reloader" Jul 01 22:34:10.538248 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538233 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy-metric" Jul 01 22:34:10.538248 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538238 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy-metric" Jul 01 22:34:10.538248 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538245 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538264 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538274 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="prom-label-proxy" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538280 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="prom-label-proxy" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538289 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="config-reloader" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538294 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="config-reloader" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538307 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dee0e561-aced-47f2-9d0c-42ab8d1224f2" containerName="console" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538312 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dee0e561-aced-47f2-9d0c-42ab8d1224f2" containerName="console" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538318 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="alertmanager" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538323 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="alertmanager" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538372 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="alertmanager" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538379 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy-metric" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538386 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="config-reloader" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538392 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538398 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dee0e561-aced-47f2-9d0c-42ab8d1224f2" containerName="console" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538403 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="kube-rbac-proxy-web" Jul 01 22:34:10.538431 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.538408 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" containerName="prom-label-proxy" Jul 01 22:34:10.541888 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.541870 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.545196 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.545178 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 01 22:34:10.545905 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.545891 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 01 22:34:10.545976 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.545891 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 01 22:34:10.546102 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.546083 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fz6tq\"" Jul 01 22:34:10.546212 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.546163 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 01 22:34:10.546461 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.546443 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 01 22:34:10.546580 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.546564 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 01 22:34:10.547389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.547361 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 01 22:34:10.558846 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.558828 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 01 22:34:10.562351 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.562330 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 01 22:34:10.585515 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.585485 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:34:10.673719 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673637 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.673719 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673672 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01fbd166-f977-495e-bdfe-54d7bf955e10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.673719 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673701 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nscfb\" (UniqueName: \"kubernetes.io/projected/01fbd166-f977-495e-bdfe-54d7bf955e10-kube-api-access-nscfb\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.673961 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673760 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.673961 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673829 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.673961 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673861 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-web-config\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.673961 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.673961 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673938 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01fbd166-f977-495e-bdfe-54d7bf955e10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.674158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.673976 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01fbd166-f977-495e-bdfe-54d7bf955e10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.674158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.674015 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01fbd166-f977-495e-bdfe-54d7bf955e10-config-out\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.674158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.674030 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/01fbd166-f977-495e-bdfe-54d7bf955e10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.674158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.674067 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.674158 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.674090 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-config-volume\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.774763 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.774728 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.774763 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.774768 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775005 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.774790 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-web-config\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775005 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.774807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775005 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.774830 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01fbd166-f977-495e-bdfe-54d7bf955e10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775005 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.774880 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01fbd166-f977-495e-bdfe-54d7bf955e10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775005 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.774957 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01fbd166-f977-495e-bdfe-54d7bf955e10-config-out\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775005 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.774984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/01fbd166-f977-495e-bdfe-54d7bf955e10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775332 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.775015 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775332 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.775041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-config-volume\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775332 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.775114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775332 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.775134 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01fbd166-f977-495e-bdfe-54d7bf955e10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775332 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.775163 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nscfb\" (UniqueName: \"kubernetes.io/projected/01fbd166-f977-495e-bdfe-54d7bf955e10-kube-api-access-nscfb\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775671 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.775644 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/01fbd166-f977-495e-bdfe-54d7bf955e10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.775939 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.775913 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01fbd166-f977-495e-bdfe-54d7bf955e10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.777926 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.777900 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01fbd166-f977-495e-bdfe-54d7bf955e10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.778035 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.777930 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.778035 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.777958 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01fbd166-f977-495e-bdfe-54d7bf955e10-config-out\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.778169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.778129 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.778236 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.778198 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.778302 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.778245 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.778514 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.778491 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01fbd166-f977-495e-bdfe-54d7bf955e10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.778589 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.778535 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.778902 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.778887 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-web-config\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.779769 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.779751 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/01fbd166-f977-495e-bdfe-54d7bf955e10-config-volume\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.784348 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.784328 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nscfb\" (UniqueName: \"kubernetes.io/projected/01fbd166-f977-495e-bdfe-54d7bf955e10-kube-api-access-nscfb\") pod \"alertmanager-main-0\" (UID: \"01fbd166-f977-495e-bdfe-54d7bf955e10\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.851377 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.851336 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 22:34:10.994200 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:10.994171 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 22:34:10.995928 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:34:10.995886 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01fbd166_f977_495e_bdfe_54d7bf955e10.slice/crio-889bb31d88a7cd4a29b0e58bc412838f417bd8d0e76332b79678c349647905f4 WatchSource:0}: Error finding container 889bb31d88a7cd4a29b0e58bc412838f417bd8d0e76332b79678c349647905f4: Status 404 returned error can't find the container with id 889bb31d88a7cd4a29b0e58bc412838f417bd8d0e76332b79678c349647905f4 Jul 01 22:34:11.464683 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:11.464651 2581 generic.go:356] "Generic (PLEG): container finished" podID="01fbd166-f977-495e-bdfe-54d7bf955e10" containerID="ed90438d98eed026ea84148457c61ef9b13769f1c8b3dd31102a8d8ee6ce7445" exitCode=0 Jul 01 22:34:11.465129 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:11.464725 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"01fbd166-f977-495e-bdfe-54d7bf955e10","Type":"ContainerDied","Data":"ed90438d98eed026ea84148457c61ef9b13769f1c8b3dd31102a8d8ee6ce7445"} Jul 01 22:34:11.465129 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:11.464786 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"01fbd166-f977-495e-bdfe-54d7bf955e10","Type":"ContainerStarted","Data":"889bb31d88a7cd4a29b0e58bc412838f417bd8d0e76332b79678c349647905f4"} Jul 01 22:34:11.669800 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:11.669757 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a1d366a-b0a6-49f2-933a-64b02259a929" path="/var/lib/kubelet/pods/8a1d366a-b0a6-49f2-933a-64b02259a929/volumes" Jul 01 22:34:12.474969 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:12.474930 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"01fbd166-f977-495e-bdfe-54d7bf955e10","Type":"ContainerStarted","Data":"1fcc93556e75c4b891f9afc00ca77d40ad2327aab6082033fe6eeb726dab0603"} Jul 01 22:34:12.475435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:12.474993 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"01fbd166-f977-495e-bdfe-54d7bf955e10","Type":"ContainerStarted","Data":"ae3c7d61214427c37b4ca7e1fff4faccd72a877e00de482873534cd54186e364"} Jul 01 22:34:12.475435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:12.475017 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"01fbd166-f977-495e-bdfe-54d7bf955e10","Type":"ContainerStarted","Data":"3d0e2b0e24528fbc01b81359ce2e29ac5c2d2f32df796d9dc9eadc4f87928f03"} Jul 01 22:34:12.475435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:12.475088 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"01fbd166-f977-495e-bdfe-54d7bf955e10","Type":"ContainerStarted","Data":"675ed46e851c3f5a84821bcf0c6080d69f45c857b651443d04bca2407ab0d1d0"} Jul 01 22:34:12.475435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:12.475111 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"01fbd166-f977-495e-bdfe-54d7bf955e10","Type":"ContainerStarted","Data":"a12e2612e845d77ff101b09945e946080831b577c58ed9c52c22d9797c410096"} Jul 01 22:34:12.475435 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:12.475134 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"01fbd166-f977-495e-bdfe-54d7bf955e10","Type":"ContainerStarted","Data":"7dff3e7f8a55a9c3f8f817a95791b29969652df7c58fd25f2d11aa388b4884a0"} Jul 01 22:34:13.100281 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.100204 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.083288849 podStartE2EDuration="3.100181311s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.892462ms" isStatefulPod=false podCreationTimestamp="2026-07-01 22:34:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:34:12.514267633 +0000 UTC m=+123.401525585" watchObservedRunningTime="2026-07-01 22:34:13.100181311 +0000 UTC m=+123.987439270" Jul 01 22:34:13.102547 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.102513 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx"] Jul 01 22:34:13.105120 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.105098 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.108803 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.108766 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 01 22:34:13.108913 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.108853 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 01 22:34:13.109110 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.109019 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 01 22:34:13.109110 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.109020 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 01 22:34:13.109272 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.109167 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 01 22:34:13.109607 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.109587 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-f49mm\"" Jul 01 22:34:13.115710 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.115642 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 01 22:34:13.122753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.122730 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx"] Jul 01 22:34:13.196139 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.196103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-federate-client-tls\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.196320 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.196144 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtssq\" (UniqueName: \"kubernetes.io/projected/42d35cee-db37-40ff-8678-f91ae118c35c-kube-api-access-jtssq\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.196320 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.196208 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-metrics-client-ca\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.196320 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.196272 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.196320 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.196310 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-telemeter-client-tls\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.196449 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.196337 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-serving-certs-ca-bundle\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.196449 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.196368 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.196449 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.196388 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-secret-telemeter-client\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.297730 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.297693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-metrics-client-ca\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.297900 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.297753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.297900 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.297790 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-telemeter-client-tls\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.297900 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.297813 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-serving-certs-ca-bundle\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.297900 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.297835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.298138 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.297966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-secret-telemeter-client\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.298138 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.298019 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-federate-client-tls\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.298138 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.298082 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtssq\" (UniqueName: \"kubernetes.io/projected/42d35cee-db37-40ff-8678-f91ae118c35c-kube-api-access-jtssq\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.298632 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.298591 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-metrics-client-ca\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.298632 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.298609 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-serving-certs-ca-bundle\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.298898 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.298878 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d35cee-db37-40ff-8678-f91ae118c35c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.300528 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.300480 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.300917 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.300894 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-telemeter-client-tls\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.300994 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.300950 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-federate-client-tls\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.300994 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.300956 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/42d35cee-db37-40ff-8678-f91ae118c35c-secret-telemeter-client\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.308232 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.308209 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtssq\" (UniqueName: \"kubernetes.io/projected/42d35cee-db37-40ff-8678-f91ae118c35c-kube-api-access-jtssq\") pod \"telemeter-client-85ddcb5c4b-bhvwx\" (UID: \"42d35cee-db37-40ff-8678-f91ae118c35c\") " pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.416609 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.416511 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" Jul 01 22:34:13.422074 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.422014 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:34:13.422484 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.422419 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="prometheus" containerID="cri-o://bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" gracePeriod=600 Jul 01 22:34:13.422583 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.422481 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" gracePeriod=600 Jul 01 22:34:13.422642 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.422614 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy" containerID="cri-o://bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" gracePeriod=600 Jul 01 22:34:13.422978 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.422710 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy-web" containerID="cri-o://da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" gracePeriod=600 Jul 01 22:34:13.422978 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.422776 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="config-reloader" containerID="cri-o://605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" gracePeriod=600 Jul 01 22:34:13.422978 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.422818 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="thanos-sidecar" containerID="cri-o://4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" gracePeriod=600 Jul 01 22:34:13.645485 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.643982 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx"] Jul 01 22:34:13.646803 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:34:13.646776 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d35cee_db37_40ff_8678_f91ae118c35c.slice/crio-f8a30452d6ee16e523ff092056eb6b66b515e20bc1ad55e0a946e8f392fddce9 WatchSource:0}: Error finding container f8a30452d6ee16e523ff092056eb6b66b515e20bc1ad55e0a946e8f392fddce9: Status 404 returned error can't find the container with id f8a30452d6ee16e523ff092056eb6b66b515e20bc1ad55e0a946e8f392fddce9 Jul 01 22:34:13.697833 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.697805 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:13.802759 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.802730 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-2\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.802759 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.802763 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-metrics-client-ca\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803012 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.802789 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-config\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803012 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.802817 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-grpc-tls\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803012 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.802939 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-config-out\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803012 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.802992 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-db\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803012 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803012 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-tls-assets\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803283 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803039 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-0\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803283 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803155 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:13.803283 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803171 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-web-config\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803283 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803204 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:13.803283 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803218 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-trusted-ca-bundle\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803519 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803308 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803519 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803338 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-serving-certs-ca-bundle\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803519 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803366 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-kubelet-serving-ca-bundle\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803519 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803394 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-tls\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803519 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803423 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-1\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803592 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-kube-rbac-proxy\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803642 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngx49\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-kube-api-access-ngx49\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803693 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803751 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803740 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-thanos-prometheus-http-client-file\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.803945 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.803780 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-metrics-client-certs\") pod \"69842bd9-bda4-4155-a486-cf50ff052a2e\" (UID: \"69842bd9-bda4-4155-a486-cf50ff052a2e\") " Jul 01 22:34:13.804327 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.804301 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.804434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.804333 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-metrics-client-ca\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.804434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.804404 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:34:13.804434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.804398 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:13.804696 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.804673 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:13.804753 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.804733 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:13.805188 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.805159 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:13.805301 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.805197 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:34:13.806246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.806216 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-config-out" (OuterVolumeSpecName: "config-out") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:34:13.806363 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.806256 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.807346 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.807320 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-config" (OuterVolumeSpecName: "config") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.807792 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.807757 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.808092 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.808038 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:34:13.808219 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.808101 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.808351 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.808233 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.808511 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.808485 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.808580 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.808525 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.808846 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.808818 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.809640 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.809620 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-kube-api-access-ngx49" (OuterVolumeSpecName: "kube-api-access-ngx49") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "kube-api-access-ngx49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:34:13.818160 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.818133 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-web-config" (OuterVolumeSpecName: "web-config") pod "69842bd9-bda4-4155-a486-cf50ff052a2e" (UID: "69842bd9-bda4-4155-a486-cf50ff052a2e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:34:13.905342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905303 2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-config\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905334 2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-grpc-tls\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905345 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-config-out\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905354 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-db\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905363 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-tls-assets\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905372 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905381 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-web-config\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905389 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905399 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905410 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905420 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905428 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905437 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/69842bd9-bda4-4155-a486-cf50ff052a2e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905446 2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-kube-rbac-proxy\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905455 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngx49\" (UniqueName: \"kubernetes.io/projected/69842bd9-bda4-4155-a486-cf50ff052a2e-kube-api-access-ngx49\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905463 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905473 2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:13.905565 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:13.905482 2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/69842bd9-bda4-4155-a486-cf50ff052a2e-secret-metrics-client-certs\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:34:14.490385 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490347 2581 generic.go:356] "Generic (PLEG): container finished" podID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" exitCode=0 Jul 01 22:34:14.490385 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490380 2581 generic.go:356] "Generic (PLEG): container finished" podID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" exitCode=0 Jul 01 22:34:14.490385 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490389 2581 generic.go:356] "Generic (PLEG): container finished" podID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" exitCode=0 Jul 01 22:34:14.490385 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490394 2581 generic.go:356] "Generic (PLEG): container finished" podID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" exitCode=0 Jul 01 22:34:14.490385 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490399 2581 generic.go:356] "Generic (PLEG): container finished" podID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" exitCode=0 Jul 01 22:34:14.490385 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490404 2581 generic.go:356] "Generic (PLEG): container finished" podID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" exitCode=0 Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490435 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490433 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerDied","Data":"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f"} Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490591 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerDied","Data":"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb"} Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490640 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerDied","Data":"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df"} Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490683 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerDied","Data":"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c"} Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490709 2581 scope.go:117] "RemoveContainer" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490727 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerDied","Data":"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f"} Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490768 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerDied","Data":"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0"} Jul 01 22:34:14.490883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.490801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"69842bd9-bda4-4155-a486-cf50ff052a2e","Type":"ContainerDied","Data":"661d362c1b8bfd56e1e6cbde8f84d3b9656b49c1408a1afc460fbda07b676888"} Jul 01 22:34:14.492130 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.492012 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" event={"ID":"42d35cee-db37-40ff-8678-f91ae118c35c","Type":"ContainerStarted","Data":"f8a30452d6ee16e523ff092056eb6b66b515e20bc1ad55e0a946e8f392fddce9"} Jul 01 22:34:14.500478 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.500453 2581 scope.go:117] "RemoveContainer" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" Jul 01 22:34:14.508581 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.508562 2581 scope.go:117] "RemoveContainer" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" Jul 01 22:34:14.516512 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.516496 2581 scope.go:117] "RemoveContainer" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" Jul 01 22:34:14.519275 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.519252 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:34:14.525495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.525473 2581 scope.go:117] "RemoveContainer" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" Jul 01 22:34:14.525603 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.525510 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:34:14.532989 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.532960 2581 scope.go:117] "RemoveContainer" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" Jul 01 22:34:14.541460 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.541435 2581 scope.go:117] "RemoveContainer" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" Jul 01 22:34:14.548080 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.548061 2581 scope.go:117] "RemoveContainer" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" Jul 01 22:34:14.548321 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:14.548304 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": container with ID starting with 6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f not found: ID does not exist" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" Jul 01 22:34:14.548369 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.548336 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f"} err="failed to get container status \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": rpc error: code = NotFound desc = could not find container \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": container with ID starting with 6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f not found: ID does not exist" Jul 01 22:34:14.548369 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.548352 2581 scope.go:117] "RemoveContainer" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" Jul 01 22:34:14.548592 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:14.548575 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": container with ID starting with bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb not found: ID does not exist" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" Jul 01 22:34:14.548644 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.548597 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb"} err="failed to get container status \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": rpc error: code = NotFound desc = could not find container \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": container with ID starting with bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb not found: ID does not exist" Jul 01 22:34:14.548644 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.548615 2581 scope.go:117] "RemoveContainer" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" Jul 01 22:34:14.548824 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:14.548808 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": container with ID starting with da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df not found: ID does not exist" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" Jul 01 22:34:14.548867 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.548829 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df"} err="failed to get container status \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": rpc error: code = NotFound desc = could not find container \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": container with ID starting with da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df not found: ID does not exist" Jul 01 22:34:14.548867 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.548843 2581 scope.go:117] "RemoveContainer" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" Jul 01 22:34:14.549068 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:14.549038 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": container with ID starting with 4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c not found: ID does not exist" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" Jul 01 22:34:14.549123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549069 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c"} err="failed to get container status \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": rpc error: code = NotFound desc = could not find container \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": container with ID starting with 4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c not found: ID does not exist" Jul 01 22:34:14.549123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549081 2581 scope.go:117] "RemoveContainer" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" Jul 01 22:34:14.549291 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:14.549276 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": container with ID starting with 605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f not found: ID does not exist" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" Jul 01 22:34:14.549334 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549294 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f"} err="failed to get container status \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": rpc error: code = NotFound desc = could not find container \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": container with ID starting with 605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f not found: ID does not exist" Jul 01 22:34:14.549334 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549306 2581 scope.go:117] "RemoveContainer" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" Jul 01 22:34:14.549543 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:14.549525 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": container with ID starting with bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0 not found: ID does not exist" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" Jul 01 22:34:14.549594 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549547 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0"} err="failed to get container status \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": rpc error: code = NotFound desc = could not find container \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": container with ID starting with bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0 not found: ID does not exist" Jul 01 22:34:14.549594 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549564 2581 scope.go:117] "RemoveContainer" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" Jul 01 22:34:14.549789 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:34:14.549773 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": container with ID starting with 70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961 not found: ID does not exist" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" Jul 01 22:34:14.549850 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549791 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961"} err="failed to get container status \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": rpc error: code = NotFound desc = could not find container \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": container with ID starting with 70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961 not found: ID does not exist" Jul 01 22:34:14.549850 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549802 2581 scope.go:117] "RemoveContainer" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" Jul 01 22:34:14.549995 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549981 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f"} err="failed to get container status \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": rpc error: code = NotFound desc = could not find container \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": container with ID starting with 6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f not found: ID does not exist" Jul 01 22:34:14.550036 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.549997 2581 scope.go:117] "RemoveContainer" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" Jul 01 22:34:14.550244 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.550225 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb"} err="failed to get container status \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": rpc error: code = NotFound desc = could not find container \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": container with ID starting with bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb not found: ID does not exist" Jul 01 22:34:14.550244 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.550244 2581 scope.go:117] "RemoveContainer" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" Jul 01 22:34:14.550457 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.550440 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df"} err="failed to get container status \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": rpc error: code = NotFound desc = could not find container \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": container with ID starting with da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df not found: ID does not exist" Jul 01 22:34:14.550495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.550460 2581 scope.go:117] "RemoveContainer" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" Jul 01 22:34:14.550700 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.550678 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c"} err="failed to get container status \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": rpc error: code = NotFound desc = could not find container \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": container with ID starting with 4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c not found: ID does not exist" Jul 01 22:34:14.550700 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.550699 2581 scope.go:117] "RemoveContainer" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" Jul 01 22:34:14.550909 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.550891 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f"} err="failed to get container status \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": rpc error: code = NotFound desc = could not find container \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": container with ID starting with 605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f not found: ID does not exist" Jul 01 22:34:14.550952 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.550910 2581 scope.go:117] "RemoveContainer" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" Jul 01 22:34:14.551163 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.551146 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0"} err="failed to get container status \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": rpc error: code = NotFound desc = could not find container \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": container with ID starting with bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0 not found: ID does not exist" Jul 01 22:34:14.551225 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.551163 2581 scope.go:117] "RemoveContainer" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" Jul 01 22:34:14.551358 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.551338 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961"} err="failed to get container status \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": rpc error: code = NotFound desc = could not find container \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": container with ID starting with 70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961 not found: ID does not exist" Jul 01 22:34:14.551426 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.551359 2581 scope.go:117] "RemoveContainer" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" Jul 01 22:34:14.551608 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.551583 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f"} err="failed to get container status \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": rpc error: code = NotFound desc = could not find container \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": container with ID starting with 6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f not found: ID does not exist" Jul 01 22:34:14.551700 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.551611 2581 scope.go:117] "RemoveContainer" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" Jul 01 22:34:14.551936 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.551911 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb"} err="failed to get container status \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": rpc error: code = NotFound desc = could not find container \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": container with ID starting with bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb not found: ID does not exist" Jul 01 22:34:14.552024 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.551938 2581 scope.go:117] "RemoveContainer" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" Jul 01 22:34:14.552249 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.552220 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df"} err="failed to get container status \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": rpc error: code = NotFound desc = could not find container \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": container with ID starting with da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df not found: ID does not exist" Jul 01 22:34:14.552342 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.552250 2581 scope.go:117] "RemoveContainer" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" Jul 01 22:34:14.552659 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.552542 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c"} err="failed to get container status \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": rpc error: code = NotFound desc = could not find container \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": container with ID starting with 4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c not found: ID does not exist" Jul 01 22:34:14.552659 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.552567 2581 scope.go:117] "RemoveContainer" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" Jul 01 22:34:14.552948 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.552928 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f"} err="failed to get container status \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": rpc error: code = NotFound desc = could not find container \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": container with ID starting with 605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f not found: ID does not exist" Jul 01 22:34:14.553021 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.552950 2581 scope.go:117] "RemoveContainer" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" Jul 01 22:34:14.553237 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553212 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0"} err="failed to get container status \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": rpc error: code = NotFound desc = could not find container \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": container with ID starting with bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0 not found: ID does not exist" Jul 01 22:34:14.553306 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553239 2581 scope.go:117] "RemoveContainer" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" Jul 01 22:34:14.553522 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553504 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961"} err="failed to get container status \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": rpc error: code = NotFound desc = could not find container \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": container with ID starting with 70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961 not found: ID does not exist" Jul 01 22:34:14.553582 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553525 2581 scope.go:117] "RemoveContainer" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" Jul 01 22:34:14.553619 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553583 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:34:14.553770 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553750 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f"} err="failed to get container status \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": rpc error: code = NotFound desc = could not find container \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": container with ID starting with 6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f not found: ID does not exist" Jul 01 22:34:14.553813 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553773 2581 scope.go:117] "RemoveContainer" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" Jul 01 22:34:14.553955 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553941 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy-thanos" Jul 01 22:34:14.553996 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553959 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy-thanos" Jul 01 22:34:14.553996 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553970 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy-web" Jul 01 22:34:14.553996 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553976 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy-web" Jul 01 22:34:14.553996 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553983 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy" Jul 01 22:34:14.553996 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553988 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554005 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="prometheus" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554014 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="prometheus" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554028 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="config-reloader" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554033 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="config-reloader" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554038 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="thanos-sidecar" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554045 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="thanos-sidecar" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554075 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="init-config-reloader" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554080 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="init-config-reloader" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554132 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="config-reloader" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.553982 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb"} err="failed to get container status \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": rpc error: code = NotFound desc = could not find container \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": container with ID starting with bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb not found: ID does not exist" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554142 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="prometheus" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554153 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="thanos-sidecar" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554153 2581 scope.go:117] "RemoveContainer" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554158 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy-thanos" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554165 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy" Jul 01 22:34:14.554169 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.554173 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" containerName="kube-rbac-proxy-web" Jul 01 22:34:14.555204 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.555013 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df"} err="failed to get container status \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": rpc error: code = NotFound desc = could not find container \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": container with ID starting with da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df not found: ID does not exist" Jul 01 22:34:14.555204 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.555041 2581 scope.go:117] "RemoveContainer" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" Jul 01 22:34:14.557670 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.556640 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c"} err="failed to get container status \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": rpc error: code = NotFound desc = could not find container \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": container with ID starting with 4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c not found: ID does not exist" Jul 01 22:34:14.557670 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.556663 2581 scope.go:117] "RemoveContainer" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" Jul 01 22:34:14.557670 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.557141 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f"} err="failed to get container status \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": rpc error: code = NotFound desc = could not find container \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": container with ID starting with 605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f not found: ID does not exist" Jul 01 22:34:14.557670 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.557165 2581 scope.go:117] "RemoveContainer" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" Jul 01 22:34:14.557865 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.557706 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0"} err="failed to get container status \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": rpc error: code = NotFound desc = could not find container \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": container with ID starting with bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0 not found: ID does not exist" Jul 01 22:34:14.557865 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.557728 2581 scope.go:117] "RemoveContainer" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" Jul 01 22:34:14.558332 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.558311 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961"} err="failed to get container status \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": rpc error: code = NotFound desc = could not find container \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": container with ID starting with 70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961 not found: ID does not exist" Jul 01 22:34:14.558391 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.558332 2581 scope.go:117] "RemoveContainer" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" Jul 01 22:34:14.558611 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.558592 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f"} err="failed to get container status \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": rpc error: code = NotFound desc = could not find container \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": container with ID starting with 6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f not found: ID does not exist" Jul 01 22:34:14.558699 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.558613 2581 scope.go:117] "RemoveContainer" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" Jul 01 22:34:14.558861 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.558841 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb"} err="failed to get container status \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": rpc error: code = NotFound desc = could not find container \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": container with ID starting with bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb not found: ID does not exist" Jul 01 22:34:14.558861 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.558860 2581 scope.go:117] "RemoveContainer" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" Jul 01 22:34:14.559100 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559082 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df"} err="failed to get container status \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": rpc error: code = NotFound desc = could not find container \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": container with ID starting with da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df not found: ID does not exist" Jul 01 22:34:14.559161 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559100 2581 scope.go:117] "RemoveContainer" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" Jul 01 22:34:14.559329 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559310 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c"} err="failed to get container status \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": rpc error: code = NotFound desc = could not find container \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": container with ID starting with 4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c not found: ID does not exist" Jul 01 22:34:14.559329 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559329 2581 scope.go:117] "RemoveContainer" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" Jul 01 22:34:14.559504 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559489 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f"} err="failed to get container status \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": rpc error: code = NotFound desc = could not find container \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": container with ID starting with 605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f not found: ID does not exist" Jul 01 22:34:14.559539 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559504 2581 scope.go:117] "RemoveContainer" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" Jul 01 22:34:14.559687 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559671 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.559823 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559673 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0"} err="failed to get container status \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": rpc error: code = NotFound desc = could not find container \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": container with ID starting with bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0 not found: ID does not exist" Jul 01 22:34:14.559923 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.559830 2581 scope.go:117] "RemoveContainer" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" Jul 01 22:34:14.560281 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.560224 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961"} err="failed to get container status \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": rpc error: code = NotFound desc = could not find container \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": container with ID starting with 70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961 not found: ID does not exist" Jul 01 22:34:14.560281 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.560252 2581 scope.go:117] "RemoveContainer" containerID="6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f" Jul 01 22:34:14.560504 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.560484 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f"} err="failed to get container status \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": rpc error: code = NotFound desc = could not find container \"6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f\": container with ID starting with 6ff96b8f24d58dbce0aedc9317917d0907b4fbb795282d3fe29c1b27584f2d1f not found: ID does not exist" Jul 01 22:34:14.560504 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.560504 2581 scope.go:117] "RemoveContainer" containerID="bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb" Jul 01 22:34:14.560872 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.560852 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb"} err="failed to get container status \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": rpc error: code = NotFound desc = could not find container \"bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb\": container with ID starting with bac98ad7d22a6c3a8e3ba078d004abcab1f70e5f1a18fb8c5a569a594f3269bb not found: ID does not exist" Jul 01 22:34:14.560872 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.560872 2581 scope.go:117] "RemoveContainer" containerID="da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df" Jul 01 22:34:14.561173 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.561153 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df"} err="failed to get container status \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": rpc error: code = NotFound desc = could not find container \"da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df\": container with ID starting with da53ab4565b1c745e208d60c0d85a64c7ae17108eee418c343afce0385ff43df not found: ID does not exist" Jul 01 22:34:14.561173 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.561172 2581 scope.go:117] "RemoveContainer" containerID="4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c" Jul 01 22:34:14.561432 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.561414 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c"} err="failed to get container status \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": rpc error: code = NotFound desc = could not find container \"4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c\": container with ID starting with 4f7d2d6e5b6b877bbf7ce4d5cc81a98ac4ee3caa3115381c45f871fa6696927c not found: ID does not exist" Jul 01 22:34:14.561500 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.561434 2581 scope.go:117] "RemoveContainer" containerID="605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f" Jul 01 22:34:14.561736 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.561713 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f"} err="failed to get container status \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": rpc error: code = NotFound desc = could not find container \"605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f\": container with ID starting with 605ac930b4f3d4c483defb7eefb1c4d2ab9cd8862f14900594bd4b6fb6de6b9f not found: ID does not exist" Jul 01 22:34:14.561773 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.561740 2581 scope.go:117] "RemoveContainer" containerID="bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0" Jul 01 22:34:14.561999 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.561978 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0"} err="failed to get container status \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": rpc error: code = NotFound desc = could not find container \"bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0\": container with ID starting with bee18ce78e00c5405c1272ef18b3e5af6500e37f4de6f4dbdfe362ba5d520fd0 not found: ID does not exist" Jul 01 22:34:14.562073 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.562001 2581 scope.go:117] "RemoveContainer" containerID="70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961" Jul 01 22:34:14.562257 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.562232 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961"} err="failed to get container status \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": rpc error: code = NotFound desc = could not find container \"70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961\": container with ID starting with 70595764a476a16ac1c58ddf0f92634bd76b531b8c59e2f7ec54c72c1c697961 not found: ID does not exist" Jul 01 22:34:14.565941 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.565920 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 22:34:14.566045 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.565983 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 22:34:14.566458 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.566426 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 22:34:14.566572 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.566550 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dtuc3vhh28gal\"" Jul 01 22:34:14.566684 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.566625 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 22:34:14.566815 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.566798 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 22:34:14.566899 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.566823 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wvbh2\"" Jul 01 22:34:14.567776 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.567751 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 22:34:14.570705 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.569083 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 22:34:14.570705 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.569633 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 22:34:14.570705 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.569677 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 22:34:14.570705 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.569800 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 22:34:14.570705 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.569938 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 22:34:14.570705 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.570665 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 22:34:14.573607 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.573587 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 22:34:14.577120 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.577101 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 22:34:14.579455 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.579437 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:34:14.713582 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.713582 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713585 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-web-config\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713610 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713634 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713660 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713692 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-config-out\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713818 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713925 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.713981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714041 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714460 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714078 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714460 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714105 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-config\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714460 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714126 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714460 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714149 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714460 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714170 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714460 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714190 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.714460 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.714241 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcmfj\" (UniqueName: \"kubernetes.io/projected/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-kube-api-access-kcmfj\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815037 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.814940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815037 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.814987 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-web-config\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815037 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815011 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815037 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-config-out\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815134 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815170 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815219 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815244 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815366 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-config\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815934 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815934 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815934 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815495 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815934 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815521 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815934 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815554 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcmfj\" (UniqueName: \"kubernetes.io/projected/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-kube-api-access-kcmfj\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.815934 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.815899 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.816294 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.816175 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.816842 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.816540 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.818193 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.818157 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.818308 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.818291 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-web-config\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.818655 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.818532 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.818994 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.818863 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.818994 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.818928 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.818994 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.818938 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.819306 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.819253 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.819419 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.819324 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-config\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.819479 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.819415 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.819655 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.819598 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.819887 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.819860 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.819980 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.819884 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-config-out\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.820546 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.820525 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.821037 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.821018 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.821550 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.821529 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.821725 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.821710 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.825084 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.825063 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcmfj\" (UniqueName: \"kubernetes.io/projected/3c055538-5eb9-499a-8c2c-dc5ab88bb47d-kube-api-access-kcmfj\") pod \"prometheus-k8s-0\" (UID: \"3c055538-5eb9-499a-8c2c-dc5ab88bb47d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:14.871018 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:14.870986 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:34:15.016320 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:15.016286 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 22:34:15.020635 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:34:15.020604 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c055538_5eb9_499a_8c2c_dc5ab88bb47d.slice/crio-762aa54b2c2b95669f70ad208d40babb4126156dc5920ada1658a04ea8159d6f WatchSource:0}: Error finding container 762aa54b2c2b95669f70ad208d40babb4126156dc5920ada1658a04ea8159d6f: Status 404 returned error can't find the container with id 762aa54b2c2b95669f70ad208d40babb4126156dc5920ada1658a04ea8159d6f Jul 01 22:34:15.499062 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:15.499005 2581 generic.go:356] "Generic (PLEG): container finished" podID="3c055538-5eb9-499a-8c2c-dc5ab88bb47d" containerID="052e4430b274b89c4e3219050b3e3cabfdaa7548626f015ed90072ec7c806377" exitCode=0 Jul 01 22:34:15.499262 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:15.499094 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c055538-5eb9-499a-8c2c-dc5ab88bb47d","Type":"ContainerDied","Data":"052e4430b274b89c4e3219050b3e3cabfdaa7548626f015ed90072ec7c806377"} Jul 01 22:34:15.499262 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:15.499197 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c055538-5eb9-499a-8c2c-dc5ab88bb47d","Type":"ContainerStarted","Data":"762aa54b2c2b95669f70ad208d40babb4126156dc5920ada1658a04ea8159d6f"} Jul 01 22:34:15.664942 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:15.664906 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69842bd9-bda4-4155-a486-cf50ff052a2e" path="/var/lib/kubelet/pods/69842bd9-bda4-4155-a486-cf50ff052a2e/volumes" Jul 01 22:34:16.507889 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.507849 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" event={"ID":"42d35cee-db37-40ff-8678-f91ae118c35c","Type":"ContainerStarted","Data":"35e6a0420dce0bad431710733b8998bbf13437f4944d7671c0f4fbeded6cfebb"} Jul 01 22:34:16.508360 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.507951 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" event={"ID":"42d35cee-db37-40ff-8678-f91ae118c35c","Type":"ContainerStarted","Data":"934d9addf4fbbc66b2413c2cef3ca83209271cbfa0ada4cb0de5738f7c25e95b"} Jul 01 22:34:16.508360 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.507980 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" event={"ID":"42d35cee-db37-40ff-8678-f91ae118c35c","Type":"ContainerStarted","Data":"a52f206173eeab7ade9bc06f558a4f334667e0bb6fdfce9cf591c4fcada7426e"} Jul 01 22:34:16.511133 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.511107 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c055538-5eb9-499a-8c2c-dc5ab88bb47d","Type":"ContainerStarted","Data":"c26ac851403ed728d8c3a1593b7142707c8d8fed189ec8e58c9afab9c4c8c2da"} Jul 01 22:34:16.511271 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.511174 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c055538-5eb9-499a-8c2c-dc5ab88bb47d","Type":"ContainerStarted","Data":"d96e94e255c9cc5cfedbe2d0a9b66e4572df76e3d60168fffefb830d187cbc1a"} Jul 01 22:34:16.511271 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.511222 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c055538-5eb9-499a-8c2c-dc5ab88bb47d","Type":"ContainerStarted","Data":"7174375a3caad3ecdf3f7823e4d6603248df3e1eb69a4be8763e6d133eca7a4a"} Jul 01 22:34:16.511271 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.511259 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c055538-5eb9-499a-8c2c-dc5ab88bb47d","Type":"ContainerStarted","Data":"610fb9f1bafa755bf41582e57531142720d8735332f2939a66ccad765d57b085"} Jul 01 22:34:16.511429 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.511301 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c055538-5eb9-499a-8c2c-dc5ab88bb47d","Type":"ContainerStarted","Data":"f0b8f29eb931a35b715c1360756abafef8e7d5d3d6a929007c971d70e8f12d3b"} Jul 01 22:34:16.511429 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.511341 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c055538-5eb9-499a-8c2c-dc5ab88bb47d","Type":"ContainerStarted","Data":"0b4a12020de165ccf121aabd9aa9ec86ce86227b2a3ecd4006efbb2ee81e0ee3"} Jul 01 22:34:16.559089 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.559019 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85ddcb5c4b-bhvwx" podStartSLOduration=1.5197091409999999 podStartE2EDuration="3.559003596s" totalImagesPullingTime="2.039294455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:34:16.555137389 +0000 UTC m=+127.442395366" watchObservedRunningTime="2026-07-01 22:34:16.559003596 +0000 UTC m=+127.446261596" Jul 01 22:34:16.617015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:16.616958 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.597649832 podStartE2EDuration="2.616939495s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.289663ms" isStatefulPod=false podCreationTimestamp="2026-07-01 22:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:34:16.61564794 +0000 UTC m=+127.502905919" watchObservedRunningTime="2026-07-01 22:34:16.616939495 +0000 UTC m=+127.504197452" Jul 01 22:34:19.871322 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:34:19.871276 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:35:14.872115 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:35:14.872080 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:35:14.887982 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:35:14.887955 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:35:15.764777 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:35:15.764750 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 22:36:58.505813 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.505732 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d"] Jul 01 22:36:58.508907 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.508890 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:36:58.512214 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.512188 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 01 22:36:58.512341 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.512188 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 01 22:36:58.513877 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.513854 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-lx2f4\"" Jul 01 22:36:58.518127 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.518101 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d"] Jul 01 22:36:58.614327 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.614288 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a29c324d-7191-46e2-b275-19bab51d9e2d-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-ndf9d\" (UID: \"a29c324d-7191-46e2-b275-19bab51d9e2d\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:36:58.614507 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.614368 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mzzk\" (UniqueName: \"kubernetes.io/projected/a29c324d-7191-46e2-b275-19bab51d9e2d-kube-api-access-4mzzk\") pod \"cert-manager-webhook-7dc76cbb94-ndf9d\" (UID: \"a29c324d-7191-46e2-b275-19bab51d9e2d\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:36:58.715761 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.715722 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a29c324d-7191-46e2-b275-19bab51d9e2d-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-ndf9d\" (UID: \"a29c324d-7191-46e2-b275-19bab51d9e2d\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:36:58.715964 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.715810 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mzzk\" (UniqueName: \"kubernetes.io/projected/a29c324d-7191-46e2-b275-19bab51d9e2d-kube-api-access-4mzzk\") pod \"cert-manager-webhook-7dc76cbb94-ndf9d\" (UID: \"a29c324d-7191-46e2-b275-19bab51d9e2d\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:36:58.724771 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.724741 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a29c324d-7191-46e2-b275-19bab51d9e2d-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-ndf9d\" (UID: \"a29c324d-7191-46e2-b275-19bab51d9e2d\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:36:58.724995 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.724978 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mzzk\" (UniqueName: \"kubernetes.io/projected/a29c324d-7191-46e2-b275-19bab51d9e2d-kube-api-access-4mzzk\") pod \"cert-manager-webhook-7dc76cbb94-ndf9d\" (UID: \"a29c324d-7191-46e2-b275-19bab51d9e2d\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:36:58.834283 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.834183 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:36:58.967797 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:58.967762 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d"] Jul 01 22:36:58.970606 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:36:58.970582 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda29c324d_7191_46e2_b275_19bab51d9e2d.slice/crio-255e259e1189ba2554abe087475bcb64d7b1775182b274571a0cd9e6502ac78a WatchSource:0}: Error finding container 255e259e1189ba2554abe087475bcb64d7b1775182b274571a0cd9e6502ac78a: Status 404 returned error can't find the container with id 255e259e1189ba2554abe087475bcb64d7b1775182b274571a0cd9e6502ac78a Jul 01 22:36:59.171110 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:36:59.171013 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" event={"ID":"a29c324d-7191-46e2-b275-19bab51d9e2d","Type":"ContainerStarted","Data":"255e259e1189ba2554abe087475bcb64d7b1775182b274571a0cd9e6502ac78a"} Jul 01 22:37:03.190021 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:03.189982 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" event={"ID":"a29c324d-7191-46e2-b275-19bab51d9e2d","Type":"ContainerStarted","Data":"1fbc2bb72cb94399be058aa1d35b5aea850a54e4932f7e1e210f29c4c48b3cc0"} Jul 01 22:37:03.190544 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:03.190107 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:37:03.210045 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:03.209995 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" podStartSLOduration=1.896677939 podStartE2EDuration="5.209979094s" totalImagesPullingTime="3.313301155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:37:03.207922899 +0000 UTC m=+294.095180856" watchObservedRunningTime="2026-07-01 22:37:03.209979094 +0000 UTC m=+294.097237051" Jul 01 22:37:06.596709 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.596671 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-2bmh2"] Jul 01 22:37:06.600259 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.600242 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-2bmh2" Jul 01 22:37:06.603150 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.603128 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xbbq4\"" Jul 01 22:37:06.609491 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.609471 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-2bmh2"] Jul 01 22:37:06.691499 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.691457 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkq5j\" (UniqueName: \"kubernetes.io/projected/be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2-kube-api-access-vkq5j\") pod \"cert-manager-59c77684cc-2bmh2\" (UID: \"be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2\") " pod="cert-manager/cert-manager-59c77684cc-2bmh2" Jul 01 22:37:06.691690 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.691512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2-bound-sa-token\") pod \"cert-manager-59c77684cc-2bmh2\" (UID: \"be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2\") " pod="cert-manager/cert-manager-59c77684cc-2bmh2" Jul 01 22:37:06.792824 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.792790 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkq5j\" (UniqueName: \"kubernetes.io/projected/be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2-kube-api-access-vkq5j\") pod \"cert-manager-59c77684cc-2bmh2\" (UID: \"be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2\") " pod="cert-manager/cert-manager-59c77684cc-2bmh2" Jul 01 22:37:06.792985 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.792842 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2-bound-sa-token\") pod \"cert-manager-59c77684cc-2bmh2\" (UID: \"be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2\") " pod="cert-manager/cert-manager-59c77684cc-2bmh2" Jul 01 22:37:06.803300 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.803273 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2-bound-sa-token\") pod \"cert-manager-59c77684cc-2bmh2\" (UID: \"be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2\") " pod="cert-manager/cert-manager-59c77684cc-2bmh2" Jul 01 22:37:06.803424 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.803304 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkq5j\" (UniqueName: \"kubernetes.io/projected/be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2-kube-api-access-vkq5j\") pod \"cert-manager-59c77684cc-2bmh2\" (UID: \"be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2\") " pod="cert-manager/cert-manager-59c77684cc-2bmh2" Jul 01 22:37:06.909842 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:06.909755 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-2bmh2" Jul 01 22:37:07.042768 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:07.042739 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-2bmh2"] Jul 01 22:37:07.045394 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:37:07.045362 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe4d9db7_6bc0_4f58_93a7_6ea3f7f545d2.slice/crio-31135c613665f0a2aa9912bff25641b055aaac7e4466813e57f26651a156fc30 WatchSource:0}: Error finding container 31135c613665f0a2aa9912bff25641b055aaac7e4466813e57f26651a156fc30: Status 404 returned error can't find the container with id 31135c613665f0a2aa9912bff25641b055aaac7e4466813e57f26651a156fc30 Jul 01 22:37:07.212369 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:07.212333 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-2bmh2" event={"ID":"be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2","Type":"ContainerStarted","Data":"6a27dab4cbf50ccbe31a764a808779491a938150b21ce44a850c58ab34479105"} Jul 01 22:37:07.212534 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:07.212379 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-2bmh2" event={"ID":"be4d9db7-6bc0-4f58-93a7-6ea3f7f545d2","Type":"ContainerStarted","Data":"31135c613665f0a2aa9912bff25641b055aaac7e4466813e57f26651a156fc30"} Jul 01 22:37:07.233242 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:07.233186 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-2bmh2" podStartSLOduration=1.233171809 podStartE2EDuration="1.233171809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:37:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:37:07.231756362 +0000 UTC m=+298.119014320" watchObservedRunningTime="2026-07-01 22:37:07.233171809 +0000 UTC m=+298.120429766" Jul 01 22:37:09.196513 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:09.196486 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-ndf9d" Jul 01 22:37:15.017040 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.017006 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn"] Jul 01 22:37:15.023521 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.023503 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" Jul 01 22:37:15.026247 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.026218 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:37:15.026345 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.026218 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 01 22:37:15.027662 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.027619 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-tzxcd\"" Jul 01 22:37:15.029350 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.029328 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn"] Jul 01 22:37:15.167837 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.167805 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v9k4\" (UniqueName: \"kubernetes.io/projected/2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9-kube-api-access-9v9k4\") pod \"openshift-lws-operator-fd8ccff4c-9hvrn\" (UID: \"2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" Jul 01 22:37:15.168009 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.167866 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9hvrn\" (UID: \"2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" Jul 01 22:37:15.268454 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.268349 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9hvrn\" (UID: \"2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" Jul 01 22:37:15.268454 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.268452 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9k4\" (UniqueName: \"kubernetes.io/projected/2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9-kube-api-access-9v9k4\") pod \"openshift-lws-operator-fd8ccff4c-9hvrn\" (UID: \"2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" Jul 01 22:37:15.268742 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.268721 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9hvrn\" (UID: \"2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" Jul 01 22:37:15.279413 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.279390 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v9k4\" (UniqueName: \"kubernetes.io/projected/2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9-kube-api-access-9v9k4\") pod \"openshift-lws-operator-fd8ccff4c-9hvrn\" (UID: \"2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" Jul 01 22:37:15.333415 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.333380 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" Jul 01 22:37:15.463238 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.463207 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn"] Jul 01 22:37:15.465569 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:37:15.465543 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fb8a89f_ce78_43b4_b27a_176ab1e7a5a9.slice/crio-26c95b91d04cd89b8234c9e8566bd6956df3c915c67d61bada7dd9335e98ef83 WatchSource:0}: Error finding container 26c95b91d04cd89b8234c9e8566bd6956df3c915c67d61bada7dd9335e98ef83: Status 404 returned error can't find the container with id 26c95b91d04cd89b8234c9e8566bd6956df3c915c67d61bada7dd9335e98ef83 Jul 01 22:37:15.467284 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:15.467264 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 22:37:16.255204 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:16.255168 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" event={"ID":"2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9","Type":"ContainerStarted","Data":"26c95b91d04cd89b8234c9e8566bd6956df3c915c67d61bada7dd9335e98ef83"} Jul 01 22:37:19.270672 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:19.270629 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" event={"ID":"2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9","Type":"ContainerStarted","Data":"a1309c752d4cc31e2b7840b8dbecd165eb839d7ac0a62428f4e63556d756c366"} Jul 01 22:37:19.292039 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:19.291978 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9hvrn" podStartSLOduration=2.464190947 podStartE2EDuration="5.291955932s" totalImagesPullingTime="2.827764985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:37:19.290259827 +0000 UTC m=+310.177517784" watchObservedRunningTime="2026-07-01 22:37:19.291955932 +0000 UTC m=+310.179213890" Jul 01 22:37:35.279987 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.279952 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf"] Jul 01 22:37:35.282794 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.282777 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.294369 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.294340 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 01 22:37:35.294369 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.294364 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 01 22:37:35.294552 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.294376 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 01 22:37:35.294552 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.294441 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 01 22:37:35.294552 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.294529 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8mk5x\"" Jul 01 22:37:35.339364 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.339331 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf"] Jul 01 22:37:35.447029 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.446983 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a911cef9-0bf8-40a5-94dc-279ab239559e-tmp\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.447230 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.447074 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a911cef9-0bf8-40a5-94dc-279ab239559e-manifests\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.447230 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.447095 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a911cef9-0bf8-40a5-94dc-279ab239559e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.447230 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.447182 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5f6n\" (UniqueName: \"kubernetes.io/projected/a911cef9-0bf8-40a5-94dc-279ab239559e-kube-api-access-b5f6n\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.447335 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.447258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a911cef9-0bf8-40a5-94dc-279ab239559e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.548626 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.548528 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a911cef9-0bf8-40a5-94dc-279ab239559e-tmp\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.548626 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.548574 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a911cef9-0bf8-40a5-94dc-279ab239559e-manifests\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.548626 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.548594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a911cef9-0bf8-40a5-94dc-279ab239559e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.548890 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.548634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5f6n\" (UniqueName: \"kubernetes.io/projected/a911cef9-0bf8-40a5-94dc-279ab239559e-kube-api-access-b5f6n\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.548890 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.548654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a911cef9-0bf8-40a5-94dc-279ab239559e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.548992 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.548949 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a911cef9-0bf8-40a5-94dc-279ab239559e-manifests\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.550929 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.550905 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a911cef9-0bf8-40a5-94dc-279ab239559e-tmp\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.551216 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.551198 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a911cef9-0bf8-40a5-94dc-279ab239559e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.551285 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.551237 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a911cef9-0bf8-40a5-94dc-279ab239559e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.578168 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.578136 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5f6n\" (UniqueName: \"kubernetes.io/projected/a911cef9-0bf8-40a5-94dc-279ab239559e-kube-api-access-b5f6n\") pod \"opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf\" (UID: \"a911cef9-0bf8-40a5-94dc-279ab239559e\") " pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.593021 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.592996 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:35.772468 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:35.772444 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf"] Jul 01 22:37:35.774789 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:37:35.774762 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda911cef9_0bf8_40a5_94dc_279ab239559e.slice/crio-129a07e2596224cb9798feafada0fafa7a5aaff30abe2782d52392b7eea74b81 WatchSource:0}: Error finding container 129a07e2596224cb9798feafada0fafa7a5aaff30abe2782d52392b7eea74b81: Status 404 returned error can't find the container with id 129a07e2596224cb9798feafada0fafa7a5aaff30abe2782d52392b7eea74b81 Jul 01 22:37:36.347703 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:36.347661 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" event={"ID":"a911cef9-0bf8-40a5-94dc-279ab239559e","Type":"ContainerStarted","Data":"129a07e2596224cb9798feafada0fafa7a5aaff30abe2782d52392b7eea74b81"} Jul 01 22:37:39.362732 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:39.362697 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" event={"ID":"a911cef9-0bf8-40a5-94dc-279ab239559e","Type":"ContainerStarted","Data":"1349bd934003a04126fdde795b1b64bbd7d36ec757aabb77e842544192daf4de"} Jul 01 22:37:40.368955 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:40.368907 2581 generic.go:356] "Generic (PLEG): container finished" podID="a911cef9-0bf8-40a5-94dc-279ab239559e" containerID="1349bd934003a04126fdde795b1b64bbd7d36ec757aabb77e842544192daf4de" exitCode=0 Jul 01 22:37:40.369371 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:40.368995 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" event={"ID":"a911cef9-0bf8-40a5-94dc-279ab239559e","Type":"ContainerDied","Data":"1349bd934003a04126fdde795b1b64bbd7d36ec757aabb77e842544192daf4de"} Jul 01 22:37:41.376365 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:41.376323 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" event={"ID":"a911cef9-0bf8-40a5-94dc-279ab239559e","Type":"ContainerStarted","Data":"31179d683cf4e0cb802c18a00b39e87191b5473b9e8f8e840e674516cee58fda"} Jul 01 22:37:41.376801 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:41.376457 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:41.406376 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:41.406320 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" podStartSLOduration=2.037917275 podStartE2EDuration="6.40630428s" totalImagesPullingTime="3.214824473s" totalInitContainerRuntime="1.153562532s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:37:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:37:41.404486993 +0000 UTC m=+332.291744963" watchObservedRunningTime="2026-07-01 22:37:41.40630428 +0000 UTC m=+332.293562239" Jul 01 22:37:52.382492 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:52.382462 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf" Jul 01 22:37:55.043972 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.043937 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d78589cfd-k6777"] Jul 01 22:37:55.056133 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.056104 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.083633 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.083602 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 01 22:37:55.083783 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.083620 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7f82q\"" Jul 01 22:37:55.083783 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.083656 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 01 22:37:55.085354 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.085332 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d78589cfd-k6777"] Jul 01 22:37:55.086658 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.086642 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 01 22:37:55.117292 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.117266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ba660d3b-e069-4fa4-a480-8cf55a4122df-manager-config\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.117292 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.117295 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba660d3b-e069-4fa4-a480-8cf55a4122df-cert\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.117496 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.117320 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q74lg\" (UniqueName: \"kubernetes.io/projected/ba660d3b-e069-4fa4-a480-8cf55a4122df-kube-api-access-q74lg\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.117496 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.117356 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba660d3b-e069-4fa4-a480-8cf55a4122df-metrics-cert\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.218495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.218458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ba660d3b-e069-4fa4-a480-8cf55a4122df-manager-config\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.218495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.218498 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba660d3b-e069-4fa4-a480-8cf55a4122df-cert\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.218757 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.218529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q74lg\" (UniqueName: \"kubernetes.io/projected/ba660d3b-e069-4fa4-a480-8cf55a4122df-kube-api-access-q74lg\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.218757 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.218560 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba660d3b-e069-4fa4-a480-8cf55a4122df-metrics-cert\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.219229 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.219209 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ba660d3b-e069-4fa4-a480-8cf55a4122df-manager-config\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.221087 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.221039 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba660d3b-e069-4fa4-a480-8cf55a4122df-cert\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.221189 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.221128 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba660d3b-e069-4fa4-a480-8cf55a4122df-metrics-cert\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.248980 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.248948 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q74lg\" (UniqueName: \"kubernetes.io/projected/ba660d3b-e069-4fa4-a480-8cf55a4122df-kube-api-access-q74lg\") pod \"lws-controller-manager-d78589cfd-k6777\" (UID: \"ba660d3b-e069-4fa4-a480-8cf55a4122df\") " pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.365580 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.365492 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:55.550608 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:55.550582 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d78589cfd-k6777"] Jul 01 22:37:55.550957 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:37:55.550933 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba660d3b_e069_4fa4_a480_8cf55a4122df.slice/crio-9c16b96724696e081f3760d7a2f3e08b123fbe891224f48e235cf2eda733e735 WatchSource:0}: Error finding container 9c16b96724696e081f3760d7a2f3e08b123fbe891224f48e235cf2eda733e735: Status 404 returned error can't find the container with id 9c16b96724696e081f3760d7a2f3e08b123fbe891224f48e235cf2eda733e735 Jul 01 22:37:56.449076 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:56.449009 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" event={"ID":"ba660d3b-e069-4fa4-a480-8cf55a4122df","Type":"ContainerStarted","Data":"9c16b96724696e081f3760d7a2f3e08b123fbe891224f48e235cf2eda733e735"} Jul 01 22:37:57.454558 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:57.454530 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" event={"ID":"ba660d3b-e069-4fa4-a480-8cf55a4122df","Type":"ContainerStarted","Data":"32a3fd53b17f03c6b625051a125715ab19590f71426c4286fccea05fe2d26303"} Jul 01 22:37:57.454954 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:57.454591 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:37:57.479078 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:37:57.474174 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" podStartSLOduration=1.855328162 podStartE2EDuration="3.474155019s" totalImagesPullingTime="1.618826857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:37:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:37:57.472032043 +0000 UTC m=+348.359290001" watchObservedRunningTime="2026-07-01 22:37:57.474155019 +0000 UTC m=+348.361412977" Jul 01 22:38:02.896637 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.896557 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf"] Jul 01 22:38:02.899273 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.899250 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.902456 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.902434 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dzkvk\"" Jul 01 22:38:02.903035 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.903017 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 01 22:38:02.903263 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.903245 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 01 22:38:02.903763 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.903746 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 01 22:38:02.916694 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.916669 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf"] Jul 01 22:38:02.986534 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986500 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.986534 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.986827 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.986827 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986568 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.986827 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljwp8\" (UniqueName: \"kubernetes.io/projected/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-kube-api-access-ljwp8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.986827 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986698 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.986827 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986721 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.986827 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986756 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:02.986827 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:02.986776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.031576 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.031547 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t"] Jul 01 22:38:03.034123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.034107 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.037608 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.037586 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 01 22:38:03.037695 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.037587 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 01 22:38:03.037912 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.037894 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 01 22:38:03.038777 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.038763 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bfktp\"" Jul 01 22:38:03.055701 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.055354 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t"] Jul 01 22:38:03.088201 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088175 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088306 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088204 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088306 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088413 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088468 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088439 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.088507 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088480 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.088558 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088513 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.088558 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088550 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088658 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088589 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088658 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088617 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljwp8\" (UniqueName: \"kubernetes.io/projected/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-kube-api-access-ljwp8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088736 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088659 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088736 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088659 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d33a8ff-9085-4ca8-8321-47e7670617c8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.088736 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6857\" (UniqueName: \"kubernetes.io/projected/9d33a8ff-9085-4ca8-8321-47e7670617c8-kube-api-access-c6857\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.088736 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088731 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088909 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088909 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088793 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088909 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088812 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.088909 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088832 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.088909 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.088853 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.089192 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.089096 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.089192 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.089164 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.090985 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.090961 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.091624 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.091603 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.106188 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.106165 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljwp8\" (UniqueName: \"kubernetes.io/projected/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-kube-api-access-ljwp8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.111009 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.110986 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ba2d047-39d0-4586-a0fe-d047e5df6dc6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47q2rf\" (UID: \"8ba2d047-39d0-4586-a0fe-d047e5df6dc6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.189388 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.189309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.189388 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.189343 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.189563 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.189402 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.189563 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.189437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.189563 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.189460 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.189563 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.189507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d33a8ff-9085-4ca8-8321-47e7670617c8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.189563 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.189540 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6857\" (UniqueName: \"kubernetes.io/projected/9d33a8ff-9085-4ca8-8321-47e7670617c8-kube-api-access-c6857\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.190371 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.190348 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.191791 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.191770 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.191871 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.191774 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.192036 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.192013 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d33a8ff-9085-4ca8-8321-47e7670617c8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.192201 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.192184 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.207233 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.207213 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d33a8ff-9085-4ca8-8321-47e7670617c8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.208749 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.208732 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6857\" (UniqueName: \"kubernetes.io/projected/9d33a8ff-9085-4ca8-8321-47e7670617c8-kube-api-access-c6857\") pod \"istiod-openshift-gateway-94bb8fbfd-b4d9t\" (UID: \"9d33a8ff-9085-4ca8-8321-47e7670617c8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.216602 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.216588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:03.342453 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.342408 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:03.348962 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.348940 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf"] Jul 01 22:38:03.351786 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:38:03.351762 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba2d047_39d0_4586_a0fe_d047e5df6dc6.slice/crio-c09ba5fdb3e2cdc2e6b6cdc1a55bf90ac4bf3738152bee3ca8953df1b141009a WatchSource:0}: Error finding container c09ba5fdb3e2cdc2e6b6cdc1a55bf90ac4bf3738152bee3ca8953df1b141009a: Status 404 returned error can't find the container with id c09ba5fdb3e2cdc2e6b6cdc1a55bf90ac4bf3738152bee3ca8953df1b141009a Jul 01 22:38:03.481333 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.481302 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" event={"ID":"8ba2d047-39d0-4586-a0fe-d047e5df6dc6","Type":"ContainerStarted","Data":"c09ba5fdb3e2cdc2e6b6cdc1a55bf90ac4bf3738152bee3ca8953df1b141009a"} Jul 01 22:38:03.492534 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:03.492508 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t"] Jul 01 22:38:03.494025 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:38:03.494003 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d33a8ff_9085_4ca8_8321_47e7670617c8.slice/crio-70b994c34b8e96814a59c2de7df038fd02f211f57e87c5e4803af5458c333dbc WatchSource:0}: Error finding container 70b994c34b8e96814a59c2de7df038fd02f211f57e87c5e4803af5458c333dbc: Status 404 returned error can't find the container with id 70b994c34b8e96814a59c2de7df038fd02f211f57e87c5e4803af5458c333dbc Jul 01 22:38:04.488990 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:04.488948 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" event={"ID":"9d33a8ff-9085-4ca8-8321-47e7670617c8","Type":"ContainerStarted","Data":"70b994c34b8e96814a59c2de7df038fd02f211f57e87c5e4803af5458c333dbc"} Jul 01 22:38:06.889223 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:06.889180 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:38:06.889508 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:06.889255 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:38:06.892396 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:06.892366 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:38:06.892503 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:06.892436 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:38:06.892503 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:06.892478 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:38:07.515133 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:07.515089 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" event={"ID":"8ba2d047-39d0-4586-a0fe-d047e5df6dc6","Type":"ContainerStarted","Data":"5e94738082eea04f14e7b1cf9287ed97239943abe379bf2440a976f4aa914aa6"} Jul 01 22:38:07.517321 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:07.517289 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" event={"ID":"9d33a8ff-9085-4ca8-8321-47e7670617c8","Type":"ContainerStarted","Data":"ca98b99418dba1b4c5a2e3ea3476971ac8db242bd12274378bcaa4434a44f9d8"} Jul 01 22:38:07.517495 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:07.517479 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:07.519407 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:07.519361 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-b4d9t container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 01 22:38:07.519500 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:07.519445 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" podUID="9d33a8ff-9085-4ca8-8321-47e7670617c8" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 22:38:07.557625 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:07.557575 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" podStartSLOduration=2.019228725 podStartE2EDuration="5.557560797s" totalImagesPullingTime="3.538332072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:38:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:38:07.55623712 +0000 UTC m=+358.443495077" watchObservedRunningTime="2026-07-01 22:38:07.557560797 +0000 UTC m=+358.444818754" Jul 01 22:38:07.601234 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:07.601183 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" podStartSLOduration=2.208178094 podStartE2EDuration="5.601168615s" totalImagesPullingTime="3.392990521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:38:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:38:07.600779603 +0000 UTC m=+358.488037557" watchObservedRunningTime="2026-07-01 22:38:07.601168615 +0000 UTC m=+358.488426571" Jul 01 22:38:08.217712 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:08.217679 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:08.222039 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:08.222017 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:08.461798 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:08.461764 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d78589cfd-k6777" Jul 01 22:38:08.521875 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:08.521784 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:38:08.522278 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:08.522258 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b4d9t" Jul 01 22:38:08.522621 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:38:08.522605 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47q2rf" Jul 01 22:40:33.618123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.618090 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz"] Jul 01 22:40:33.620473 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.620455 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.623720 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.623700 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ctx9w\"" Jul 01 22:40:33.633636 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.633615 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz"] Jul 01 22:40:33.686986 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.686958 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8g6l\" (UniqueName: \"kubernetes.io/projected/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-kube-api-access-t8g6l\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.687130 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.686994 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.687130 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.687010 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.687130 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.687026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.687271 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.687137 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.687271 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.687166 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.687271 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.687191 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.687271 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.687234 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.687418 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.687279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788464 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8g6l\" (UniqueName: \"kubernetes.io/projected/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-kube-api-access-t8g6l\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788625 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788625 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788625 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788625 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788582 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788625 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788610 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788900 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788649 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788900 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788737 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.788900 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.789034 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788946 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.789034 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.788972 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.789180 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.789097 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.789180 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.789130 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.789706 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.789677 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.791486 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.791442 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.792310 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.792286 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.797835 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.797808 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8g6l\" (UniqueName: \"kubernetes.io/projected/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-kube-api-access-t8g6l\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.797835 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.797817 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98ac7706-38c8-4b54-b2d3-fc58c7c6805a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8trpz\" (UID: \"98ac7706-38c8-4b54-b2d3-fc58c7c6805a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:33.930637 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:33.930546 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:34.067755 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:34.067730 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz"] Jul 01 22:40:34.069627 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:40:34.069603 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ac7706_38c8_4b54_b2d3_fc58c7c6805a.slice/crio-eb6916fcf3ae4220e1b4b83b14e37dc184c5a99de051883aa7ed66b8c55aa364 WatchSource:0}: Error finding container eb6916fcf3ae4220e1b4b83b14e37dc184c5a99de051883aa7ed66b8c55aa364: Status 404 returned error can't find the container with id eb6916fcf3ae4220e1b4b83b14e37dc184c5a99de051883aa7ed66b8c55aa364 Jul 01 22:40:34.071832 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:34.071799 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:40:34.071932 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:34.071890 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:40:34.071996 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:34.071941 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:40:34.169005 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:34.168971 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" event={"ID":"98ac7706-38c8-4b54-b2d3-fc58c7c6805a","Type":"ContainerStarted","Data":"dc0d9648c30b63739cb3291885590b391361b8fe96b4c822a7b8f6331fce5cf9"} Jul 01 22:40:34.169194 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:34.169031 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" event={"ID":"98ac7706-38c8-4b54-b2d3-fc58c7c6805a","Type":"ContainerStarted","Data":"eb6916fcf3ae4220e1b4b83b14e37dc184c5a99de051883aa7ed66b8c55aa364"} Jul 01 22:40:34.194286 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:34.194179 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" podStartSLOduration=1.194159699 podStartE2EDuration="1.194159699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:40:34.192615867 +0000 UTC m=+505.079873821" watchObservedRunningTime="2026-07-01 22:40:34.194159699 +0000 UTC m=+505.081417657" Jul 01 22:40:34.931212 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:34.931186 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:35.935523 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:35.935492 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:36.178475 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:36.178439 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:40:36.179217 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:40:36.179195 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:42:15.284563 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.284526 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5744cf849f-zfqzv"] Jul 01 22:42:15.286837 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.286821 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:15.293940 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.293919 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n5b7s\"" Jul 01 22:42:15.294441 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.294428 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 01 22:42:15.294499 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.294487 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 01 22:42:15.312182 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.312152 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5744cf849f-zfqzv"] Jul 01 22:42:15.349107 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.349074 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6rlp\" (UniqueName: \"kubernetes.io/projected/72f69794-341d-43ea-8b6a-7d0f684051f2-kube-api-access-m6rlp\") pod \"maas-controller-5744cf849f-zfqzv\" (UID: \"72f69794-341d-43ea-8b6a-7d0f684051f2\") " pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:15.349229 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.349113 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f69794-341d-43ea-8b6a-7d0f684051f2-webhook-cert\") pod \"maas-controller-5744cf849f-zfqzv\" (UID: \"72f69794-341d-43ea-8b6a-7d0f684051f2\") " pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:15.450616 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.450586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rlp\" (UniqueName: \"kubernetes.io/projected/72f69794-341d-43ea-8b6a-7d0f684051f2-kube-api-access-m6rlp\") pod \"maas-controller-5744cf849f-zfqzv\" (UID: \"72f69794-341d-43ea-8b6a-7d0f684051f2\") " pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:15.450773 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.450623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f69794-341d-43ea-8b6a-7d0f684051f2-webhook-cert\") pod \"maas-controller-5744cf849f-zfqzv\" (UID: \"72f69794-341d-43ea-8b6a-7d0f684051f2\") " pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:15.453013 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.452992 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f69794-341d-43ea-8b6a-7d0f684051f2-webhook-cert\") pod \"maas-controller-5744cf849f-zfqzv\" (UID: \"72f69794-341d-43ea-8b6a-7d0f684051f2\") " pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:15.462446 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.462422 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6rlp\" (UniqueName: \"kubernetes.io/projected/72f69794-341d-43ea-8b6a-7d0f684051f2-kube-api-access-m6rlp\") pod \"maas-controller-5744cf849f-zfqzv\" (UID: \"72f69794-341d-43ea-8b6a-7d0f684051f2\") " pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:15.557626 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.557565 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76c97789cb-c594c"] Jul 01 22:42:15.559846 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.559831 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:15.577155 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.577134 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76c97789cb-c594c"] Jul 01 22:42:15.596519 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.596502 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:15.652880 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.652854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36348a95-95fb-43ae-acd1-b4ba673a27a7-webhook-cert\") pod \"maas-controller-76c97789cb-c594c\" (UID: \"36348a95-95fb-43ae-acd1-b4ba673a27a7\") " pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:15.652993 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.652884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt7tm\" (UniqueName: \"kubernetes.io/projected/36348a95-95fb-43ae-acd1-b4ba673a27a7-kube-api-access-jt7tm\") pod \"maas-controller-76c97789cb-c594c\" (UID: \"36348a95-95fb-43ae-acd1-b4ba673a27a7\") " pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:15.726154 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.726121 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5744cf849f-zfqzv"] Jul 01 22:42:15.727994 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:42:15.727969 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f69794_341d_43ea_8b6a_7d0f684051f2.slice/crio-af5eab8da50abaf6f78eff56fcc31f2b111c83fa0dc8957a01c4d4c450d11469 WatchSource:0}: Error finding container af5eab8da50abaf6f78eff56fcc31f2b111c83fa0dc8957a01c4d4c450d11469: Status 404 returned error can't find the container with id af5eab8da50abaf6f78eff56fcc31f2b111c83fa0dc8957a01c4d4c450d11469 Jul 01 22:42:15.729310 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.729293 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 22:42:15.753384 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.753352 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36348a95-95fb-43ae-acd1-b4ba673a27a7-webhook-cert\") pod \"maas-controller-76c97789cb-c594c\" (UID: \"36348a95-95fb-43ae-acd1-b4ba673a27a7\") " pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:15.753384 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.753379 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt7tm\" (UniqueName: \"kubernetes.io/projected/36348a95-95fb-43ae-acd1-b4ba673a27a7-kube-api-access-jt7tm\") pod \"maas-controller-76c97789cb-c594c\" (UID: \"36348a95-95fb-43ae-acd1-b4ba673a27a7\") " pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:15.755587 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.755563 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36348a95-95fb-43ae-acd1-b4ba673a27a7-webhook-cert\") pod \"maas-controller-76c97789cb-c594c\" (UID: \"36348a95-95fb-43ae-acd1-b4ba673a27a7\") " pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:15.767090 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.767040 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt7tm\" (UniqueName: \"kubernetes.io/projected/36348a95-95fb-43ae-acd1-b4ba673a27a7-kube-api-access-jt7tm\") pod \"maas-controller-76c97789cb-c594c\" (UID: \"36348a95-95fb-43ae-acd1-b4ba673a27a7\") " pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:15.869711 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:15.869656 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:16.021528 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:16.021492 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76c97789cb-c594c"] Jul 01 22:42:16.024624 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:42:16.024590 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36348a95_95fb_43ae_acd1_b4ba673a27a7.slice/crio-9b190763c41254a0536b19c8493e3bbcb932f12c5ff65bc8a9595900d74a7dbe WatchSource:0}: Error finding container 9b190763c41254a0536b19c8493e3bbcb932f12c5ff65bc8a9595900d74a7dbe: Status 404 returned error can't find the container with id 9b190763c41254a0536b19c8493e3bbcb932f12c5ff65bc8a9595900d74a7dbe Jul 01 22:42:16.645499 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:16.645458 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744cf849f-zfqzv" event={"ID":"72f69794-341d-43ea-8b6a-7d0f684051f2","Type":"ContainerStarted","Data":"af5eab8da50abaf6f78eff56fcc31f2b111c83fa0dc8957a01c4d4c450d11469"} Jul 01 22:42:16.648199 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:16.648169 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76c97789cb-c594c" event={"ID":"36348a95-95fb-43ae-acd1-b4ba673a27a7","Type":"ContainerStarted","Data":"9b190763c41254a0536b19c8493e3bbcb932f12c5ff65bc8a9595900d74a7dbe"} Jul 01 22:42:18.660365 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:18.660331 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76c97789cb-c594c" event={"ID":"36348a95-95fb-43ae-acd1-b4ba673a27a7","Type":"ContainerStarted","Data":"e417e4a2b6b135bd23b4547b3e83cf96444affbe98f3c2bdadb7bb3be25da75e"} Jul 01 22:42:18.660775 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:18.660385 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:18.661913 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:18.661884 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744cf849f-zfqzv" event={"ID":"72f69794-341d-43ea-8b6a-7d0f684051f2","Type":"ContainerStarted","Data":"3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3"} Jul 01 22:42:18.662026 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:18.661939 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:18.715534 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:18.715487 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5744cf849f-zfqzv" podStartSLOduration=0.988496589 podStartE2EDuration="3.715470988s" totalImagesPullingTime="2.726974399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:42:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:42:18.713965107 +0000 UTC m=+609.601223065" watchObservedRunningTime="2026-07-01 22:42:18.715470988 +0000 UTC m=+609.602728944" Jul 01 22:42:18.715682 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:18.715571 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76c97789cb-c594c" podStartSLOduration=1.274739428 podStartE2EDuration="3.715567013s" totalImagesPullingTime="2.440827585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:42:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:42:18.692984886 +0000 UTC m=+609.580242844" watchObservedRunningTime="2026-07-01 22:42:18.715567013 +0000 UTC m=+609.602824972" Jul 01 22:42:29.671897 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:29.671864 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76c97789cb-c594c" Jul 01 22:42:29.672506 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:29.671934 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:29.733595 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:29.733562 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5744cf849f-zfqzv"] Jul 01 22:42:29.733781 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:29.733738 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5744cf849f-zfqzv" podUID="72f69794-341d-43ea-8b6a-7d0f684051f2" containerName="manager" containerID="cri-o://3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3" gracePeriod=30 Jul 01 22:42:29.977358 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:29.977334 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:30.082652 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.082631 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f69794-341d-43ea-8b6a-7d0f684051f2-webhook-cert\") pod \"72f69794-341d-43ea-8b6a-7d0f684051f2\" (UID: \"72f69794-341d-43ea-8b6a-7d0f684051f2\") " Jul 01 22:42:30.082783 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.082729 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6rlp\" (UniqueName: \"kubernetes.io/projected/72f69794-341d-43ea-8b6a-7d0f684051f2-kube-api-access-m6rlp\") pod \"72f69794-341d-43ea-8b6a-7d0f684051f2\" (UID: \"72f69794-341d-43ea-8b6a-7d0f684051f2\") " Jul 01 22:42:30.084703 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.084674 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f69794-341d-43ea-8b6a-7d0f684051f2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "72f69794-341d-43ea-8b6a-7d0f684051f2" (UID: "72f69794-341d-43ea-8b6a-7d0f684051f2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:42:30.084791 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.084680 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f69794-341d-43ea-8b6a-7d0f684051f2-kube-api-access-m6rlp" (OuterVolumeSpecName: "kube-api-access-m6rlp") pod "72f69794-341d-43ea-8b6a-7d0f684051f2" (UID: "72f69794-341d-43ea-8b6a-7d0f684051f2"). InnerVolumeSpecName "kube-api-access-m6rlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:42:30.183933 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.183911 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6rlp\" (UniqueName: \"kubernetes.io/projected/72f69794-341d-43ea-8b6a-7d0f684051f2-kube-api-access-m6rlp\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:42:30.183933 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.183933 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72f69794-341d-43ea-8b6a-7d0f684051f2-webhook-cert\") on node \"ip-10-0-141-190.ec2.internal\" DevicePath \"\"" Jul 01 22:42:30.715637 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.715598 2581 generic.go:356] "Generic (PLEG): container finished" podID="72f69794-341d-43ea-8b6a-7d0f684051f2" containerID="3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3" exitCode=0 Jul 01 22:42:30.715637 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.715636 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5744cf849f-zfqzv" Jul 01 22:42:30.716188 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.715635 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744cf849f-zfqzv" event={"ID":"72f69794-341d-43ea-8b6a-7d0f684051f2","Type":"ContainerDied","Data":"3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3"} Jul 01 22:42:30.716188 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.715760 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5744cf849f-zfqzv" event={"ID":"72f69794-341d-43ea-8b6a-7d0f684051f2","Type":"ContainerDied","Data":"af5eab8da50abaf6f78eff56fcc31f2b111c83fa0dc8957a01c4d4c450d11469"} Jul 01 22:42:30.716188 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.715786 2581 scope.go:117] "RemoveContainer" containerID="3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3" Jul 01 22:42:30.724977 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.724964 2581 scope.go:117] "RemoveContainer" containerID="3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3" Jul 01 22:42:30.725238 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:42:30.725223 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3\": container with ID starting with 3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3 not found: ID does not exist" containerID="3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3" Jul 01 22:42:30.725283 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.725244 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3"} err="failed to get container status \"3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3\": rpc error: code = NotFound desc = could not find container \"3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3\": container with ID starting with 3e146f9473efdd2b6cd12572b301f54062c0fc7b4be2935334d09a0dcf812fc3 not found: ID does not exist" Jul 01 22:42:30.740076 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.740036 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5744cf849f-zfqzv"] Jul 01 22:42:30.741783 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:30.741763 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5744cf849f-zfqzv"] Jul 01 22:42:31.664543 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:31.664455 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f69794-341d-43ea-8b6a-7d0f684051f2" path="/var/lib/kubelet/pods/72f69794-341d-43ea-8b6a-7d0f684051f2/volumes" Jul 01 22:42:50.818928 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.818892 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5bb9dd6b4f-6g5t6"] Jul 01 22:42:50.819499 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.819479 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f69794-341d-43ea-8b6a-7d0f684051f2" containerName="manager" Jul 01 22:42:50.819589 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.819502 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f69794-341d-43ea-8b6a-7d0f684051f2" containerName="manager" Jul 01 22:42:50.819589 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.819580 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f69794-341d-43ea-8b6a-7d0f684051f2" containerName="manager" Jul 01 22:42:50.824204 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.824183 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.827775 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.827756 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bb5mm\"" Jul 01 22:42:50.827891 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.827831 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 01 22:42:50.850200 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.850168 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5bb9dd6b4f-6g5t6"] Jul 01 22:42:50.861849 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.861819 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bb7843e-9d76-46de-a6ff-b726250400de-openshift-service-ca\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.861987 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.861940 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g2z4\" (UniqueName: \"kubernetes.io/projected/1bb7843e-9d76-46de-a6ff-b726250400de-kube-api-access-6g2z4\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.862079 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.862012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bb7843e-9d76-46de-a6ff-b726250400de-maas-api-tls\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.963246 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.963211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bb7843e-9d76-46de-a6ff-b726250400de-openshift-service-ca\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.963404 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.963316 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g2z4\" (UniqueName: \"kubernetes.io/projected/1bb7843e-9d76-46de-a6ff-b726250400de-kube-api-access-6g2z4\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.963404 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.963376 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bb7843e-9d76-46de-a6ff-b726250400de-maas-api-tls\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.963855 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.963832 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bb7843e-9d76-46de-a6ff-b726250400de-openshift-service-ca\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.965686 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.965665 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bb7843e-9d76-46de-a6ff-b726250400de-maas-api-tls\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:50.977305 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:50.977280 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g2z4\" (UniqueName: \"kubernetes.io/projected/1bb7843e-9d76-46de-a6ff-b726250400de-kube-api-access-6g2z4\") pod \"maas-api-5bb9dd6b4f-6g5t6\" (UID: \"1bb7843e-9d76-46de-a6ff-b726250400de\") " pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:51.134118 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.134041 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:51.286221 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.286193 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5bb9dd6b4f-6g5t6"] Jul 01 22:42:51.287335 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:42:51.287307 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb7843e_9d76_46de_a6ff_b726250400de.slice/crio-4b774be76fea710bfe9e711d3838bafc25447d629bb22aa0d3d8489ed406e7c0 WatchSource:0}: Error finding container 4b774be76fea710bfe9e711d3838bafc25447d629bb22aa0d3d8489ed406e7c0: Status 404 returned error can't find the container with id 4b774be76fea710bfe9e711d3838bafc25447d629bb22aa0d3d8489ed406e7c0 Jul 01 22:42:51.727366 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.727322 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-648v7"] Jul 01 22:42:51.730265 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.730247 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:51.733012 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.732989 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 01 22:42:51.733513 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.733498 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vdpns\"" Jul 01 22:42:51.749798 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.749775 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-648v7"] Jul 01 22:42:51.771159 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.771135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx6fj\" (UniqueName: \"kubernetes.io/projected/1413feed-61f7-49f4-92c1-2c413976b7c4-kube-api-access-tx6fj\") pod \"payload-processing-7bbcb654f4-648v7\" (UID: \"1413feed-61f7-49f4-92c1-2c413976b7c4\") " pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:51.771245 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.771202 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1413feed-61f7-49f4-92c1-2c413976b7c4-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-648v7\" (UID: \"1413feed-61f7-49f4-92c1-2c413976b7c4\") " pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:51.813591 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.813557 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" event={"ID":"1bb7843e-9d76-46de-a6ff-b726250400de","Type":"ContainerStarted","Data":"4b774be76fea710bfe9e711d3838bafc25447d629bb22aa0d3d8489ed406e7c0"} Jul 01 22:42:51.818543 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.818522 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-w822m"] Jul 01 22:42:51.820855 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.820841 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:51.842630 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.842608 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-w822m"] Jul 01 22:42:51.871917 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.871890 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4d2911b-02e5-4597-8b6c-5d3c19f8b148-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-w822m\" (UID: \"d4d2911b-02e5-4597-8b6c-5d3c19f8b148\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:51.872036 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.871942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6fj\" (UniqueName: \"kubernetes.io/projected/1413feed-61f7-49f4-92c1-2c413976b7c4-kube-api-access-tx6fj\") pod \"payload-processing-7bbcb654f4-648v7\" (UID: \"1413feed-61f7-49f4-92c1-2c413976b7c4\") " pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:51.872123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.872042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng928\" (UniqueName: \"kubernetes.io/projected/d4d2911b-02e5-4597-8b6c-5d3c19f8b148-kube-api-access-ng928\") pod \"payload-pre-processing-59f49fd4f6-w822m\" (UID: \"d4d2911b-02e5-4597-8b6c-5d3c19f8b148\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:51.872123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.872094 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1413feed-61f7-49f4-92c1-2c413976b7c4-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-648v7\" (UID: \"1413feed-61f7-49f4-92c1-2c413976b7c4\") " pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:51.872893 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.872870 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1413feed-61f7-49f4-92c1-2c413976b7c4-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-648v7\" (UID: \"1413feed-61f7-49f4-92c1-2c413976b7c4\") " pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:51.883853 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.883825 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6fj\" (UniqueName: \"kubernetes.io/projected/1413feed-61f7-49f4-92c1-2c413976b7c4-kube-api-access-tx6fj\") pod \"payload-processing-7bbcb654f4-648v7\" (UID: \"1413feed-61f7-49f4-92c1-2c413976b7c4\") " pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:51.973449 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.973408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng928\" (UniqueName: \"kubernetes.io/projected/d4d2911b-02e5-4597-8b6c-5d3c19f8b148-kube-api-access-ng928\") pod \"payload-pre-processing-59f49fd4f6-w822m\" (UID: \"d4d2911b-02e5-4597-8b6c-5d3c19f8b148\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:51.973613 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.973501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4d2911b-02e5-4597-8b6c-5d3c19f8b148-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-w822m\" (UID: \"d4d2911b-02e5-4597-8b6c-5d3c19f8b148\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:51.974527 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.974465 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4d2911b-02e5-4597-8b6c-5d3c19f8b148-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-w822m\" (UID: \"d4d2911b-02e5-4597-8b6c-5d3c19f8b148\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:51.984783 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:51.984719 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng928\" (UniqueName: \"kubernetes.io/projected/d4d2911b-02e5-4597-8b6c-5d3c19f8b148-kube-api-access-ng928\") pod \"payload-pre-processing-59f49fd4f6-w822m\" (UID: \"d4d2911b-02e5-4597-8b6c-5d3c19f8b148\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:52.040189 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:52.040166 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:52.130482 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:52.130371 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:52.206802 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:52.206258 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-648v7"] Jul 01 22:42:52.299330 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:52.299305 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-w822m"] Jul 01 22:42:52.301240 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:42:52.301208 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d2911b_02e5_4597_8b6c_5d3c19f8b148.slice/crio-9ec3fc07038c75a040a7fb794ac8a473c301579080f9e0badd0d8ffb4e2eb414 WatchSource:0}: Error finding container 9ec3fc07038c75a040a7fb794ac8a473c301579080f9e0badd0d8ffb4e2eb414: Status 404 returned error can't find the container with id 9ec3fc07038c75a040a7fb794ac8a473c301579080f9e0badd0d8ffb4e2eb414 Jul 01 22:42:52.819900 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:52.819866 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" event={"ID":"1413feed-61f7-49f4-92c1-2c413976b7c4","Type":"ContainerStarted","Data":"00c5f80f6f608569abd3c029fef5f55f76a3a863718416574b03d60a4721cd13"} Jul 01 22:42:52.820962 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:52.820939 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" event={"ID":"d4d2911b-02e5-4597-8b6c-5d3c19f8b148","Type":"ContainerStarted","Data":"9ec3fc07038c75a040a7fb794ac8a473c301579080f9e0badd0d8ffb4e2eb414"} Jul 01 22:42:53.849075 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:53.842326 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" event={"ID":"1bb7843e-9d76-46de-a6ff-b726250400de","Type":"ContainerStarted","Data":"07b4e6bb6a95c8bf9fd6303ec146c1a93b61830d28bc4f21c016a29773202138"} Jul 01 22:42:53.849075 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:53.843157 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:42:53.867843 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:53.867797 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" podStartSLOduration=2.298474793 podStartE2EDuration="3.867778467s" totalImagesPullingTime="1.569303674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:42:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:42:53.865676104 +0000 UTC m=+644.752934063" watchObservedRunningTime="2026-07-01 22:42:53.867778467 +0000 UTC m=+644.755036423" Jul 01 22:42:55.867259 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:55.867124 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" event={"ID":"d4d2911b-02e5-4597-8b6c-5d3c19f8b148","Type":"ContainerStarted","Data":"31e2eda87b85292dc45e841dee46ebdc93735ffe7763f593722a578dafe0f917"} Jul 01 22:42:55.867607 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:55.867504 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:42:55.869628 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:55.869428 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" event={"ID":"1413feed-61f7-49f4-92c1-2c413976b7c4","Type":"ContainerStarted","Data":"00d358406ef4aa15c4b98bedf60dc7adc041bb5a03a582a50a80cde3afc6cf09"} Jul 01 22:42:55.869628 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:55.869573 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:42:55.887726 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:55.887687 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" podStartSLOduration=1.621946308 podStartE2EDuration="4.887673497s" totalImagesPullingTime="3.265727189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:42:55.885452525 +0000 UTC m=+646.772710501" watchObservedRunningTime="2026-07-01 22:42:55.887673497 +0000 UTC m=+646.774931453" Jul 01 22:42:55.906125 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:55.906086 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" podStartSLOduration=1.5407379780000001 podStartE2EDuration="4.906068109s" totalImagesPullingTime="3.365330131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:42:55.904952634 +0000 UTC m=+646.792210592" watchObservedRunningTime="2026-07-01 22:42:55.906068109 +0000 UTC m=+646.793326058" Jul 01 22:42:59.892666 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:59.892636 2581 generic.go:356] "Generic (PLEG): container finished" podID="98ac7706-38c8-4b54-b2d3-fc58c7c6805a" containerID="dc0d9648c30b63739cb3291885590b391361b8fe96b4c822a7b8f6331fce5cf9" exitCode=0 Jul 01 22:42:59.893089 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:59.892702 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" event={"ID":"98ac7706-38c8-4b54-b2d3-fc58c7c6805a","Type":"ContainerDied","Data":"dc0d9648c30b63739cb3291885590b391361b8fe96b4c822a7b8f6331fce5cf9"} Jul 01 22:42:59.893133 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:59.893099 2581 scope.go:117] "RemoveContainer" containerID="dc0d9648c30b63739cb3291885590b391361b8fe96b4c822a7b8f6331fce5cf9" Jul 01 22:42:59.894010 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:59.893984 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:42:59.894077 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:59.894061 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:42:59.894121 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:42:59.894097 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 01 22:43:00.875179 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:00.875149 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5bb9dd6b4f-6g5t6" Jul 01 22:43:00.900400 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:00.900367 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" event={"ID":"98ac7706-38c8-4b54-b2d3-fc58c7c6805a","Type":"ContainerStarted","Data":"515bd762c17d74958d15250cccba9562df1d658c4fb6c96be37bad94753b1262"} Jul 01 22:43:00.941074 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:00.934200 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:43:00.941074 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:00.938182 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8trpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 01 22:43:00.941074 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:00.938245 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" podUID="98ac7706-38c8-4b54-b2d3-fc58c7c6805a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 01 22:43:01.931218 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:01.931176 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8trpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 01 22:43:01.931702 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:01.931243 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" podUID="98ac7706-38c8-4b54-b2d3-fc58c7c6805a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 01 22:43:02.931719 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:02.931683 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8trpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 01 22:43:02.932324 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:02.932291 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" podUID="98ac7706-38c8-4b54-b2d3-fc58c7c6805a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 01 22:43:03.931709 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:03.931667 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8trpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 01 22:43:03.931916 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:03.931733 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" podUID="98ac7706-38c8-4b54-b2d3-fc58c7c6805a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 01 22:43:04.931090 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:04.931044 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8trpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 01 22:43:04.931249 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:04.931120 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" podUID="98ac7706-38c8-4b54-b2d3-fc58c7c6805a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 01 22:43:05.935413 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:05.935385 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:43:05.935789 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:05.935633 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:43:05.936389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:05.936370 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8trpz" Jul 01 22:43:06.877412 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:06.877381 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:43:06.877604 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:43:06.877445 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-w822m" Jul 01 22:44:25.937770 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:25.937729 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl"] Jul 01 22:44:25.941410 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:25.941389 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:25.944746 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:25.944687 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 01 22:44:25.944746 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:25.944713 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 01 22:44:25.945993 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:25.945975 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 01 22:44:25.946118 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:25.945981 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-72kwl\"" Jul 01 22:44:25.953440 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:25.953420 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl"] Jul 01 22:44:26.041992 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.041955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.042203 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.042017 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.042203 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.042074 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bcb5fd-1aca-40d2-9dff-738688081f81-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.042203 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.042111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.042203 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.042147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ppdf\" (UniqueName: \"kubernetes.io/projected/69bcb5fd-1aca-40d2-9dff-738688081f81-kube-api-access-9ppdf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.042203 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.042173 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143110 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143277 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143131 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ppdf\" (UniqueName: \"kubernetes.io/projected/69bcb5fd-1aca-40d2-9dff-738688081f81-kube-api-access-9ppdf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143277 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143163 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143277 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143202 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143277 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143277 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143267 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bcb5fd-1aca-40d2-9dff-738688081f81-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143534 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143505 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143656 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143638 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.143698 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.143689 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.145583 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.145550 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bcb5fd-1aca-40d2-9dff-738688081f81-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.145803 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.145786 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bcb5fd-1aca-40d2-9dff-738688081f81-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.152770 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.152750 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ppdf\" (UniqueName: \"kubernetes.io/projected/69bcb5fd-1aca-40d2-9dff-738688081f81-kube-api-access-9ppdf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls4cl\" (UID: \"69bcb5fd-1aca-40d2-9dff-738688081f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.253849 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.253813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:26.397276 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:26.397250 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl"] Jul 01 22:44:26.399502 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:44:26.399476 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69bcb5fd_1aca_40d2_9dff_738688081f81.slice/crio-78263d28346b48d5b85d428ed797e7501e53118e684a5276f477dbfc789cfa14 WatchSource:0}: Error finding container 78263d28346b48d5b85d428ed797e7501e53118e684a5276f477dbfc789cfa14: Status 404 returned error can't find the container with id 78263d28346b48d5b85d428ed797e7501e53118e684a5276f477dbfc789cfa14 Jul 01 22:44:27.323137 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:27.323101 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" event={"ID":"69bcb5fd-1aca-40d2-9dff-738688081f81","Type":"ContainerStarted","Data":"78263d28346b48d5b85d428ed797e7501e53118e684a5276f477dbfc789cfa14"} Jul 01 22:44:28.329164 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:28.329123 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" event={"ID":"69bcb5fd-1aca-40d2-9dff-738688081f81","Type":"ContainerStarted","Data":"2488f22892e699740c44cc5f2e67045e71b5d155b57a012974418d3c4728d887"} Jul 01 22:44:28.351452 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:28.351395 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" podStartSLOduration=2.352091316 podStartE2EDuration="3.351379504s" totalImagesPullingTime="999.288188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:44:28.349547755 +0000 UTC m=+739.236805739" watchObservedRunningTime="2026-07-01 22:44:28.351379504 +0000 UTC m=+739.238637461" Jul 01 22:44:33.934175 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:33.934140 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5"] Jul 01 22:44:33.938011 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:33.937990 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:33.940960 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:33.940935 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 01 22:44:33.953761 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:33.953734 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5"] Jul 01 22:44:34.026889 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.026855 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.027096 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.026925 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg52c\" (UniqueName: \"kubernetes.io/projected/a99dd773-8549-4e25-93a0-272c8b1cdbf2-kube-api-access-zg52c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.027096 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.026967 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.027096 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.026990 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.027281 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.027178 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.027281 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.027212 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a99dd773-8549-4e25-93a0-272c8b1cdbf2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.127863 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.127825 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.127863 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.127862 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a99dd773-8549-4e25-93a0-272c8b1cdbf2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.128148 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.127882 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.128148 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.127939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg52c\" (UniqueName: \"kubernetes.io/projected/a99dd773-8549-4e25-93a0-272c8b1cdbf2-kube-api-access-zg52c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.128148 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.127984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.128313 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.128249 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.128384 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.128310 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.128384 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.128331 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.128583 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.128564 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.130313 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.130291 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a99dd773-8549-4e25-93a0-272c8b1cdbf2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.130696 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.130674 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a99dd773-8549-4e25-93a0-272c8b1cdbf2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.136669 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.136651 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg52c\" (UniqueName: \"kubernetes.io/projected/a99dd773-8549-4e25-93a0-272c8b1cdbf2-kube-api-access-zg52c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5\" (UID: \"a99dd773-8549-4e25-93a0-272c8b1cdbf2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.248717 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.248680 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:34.382594 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:34.382570 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5"] Jul 01 22:44:34.385007 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:44:34.384974 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99dd773_8549_4e25_93a0_272c8b1cdbf2.slice/crio-2e84f398f5ccdab653025f636579112925724b0ded8e27779d741c86c62ecd1e WatchSource:0}: Error finding container 2e84f398f5ccdab653025f636579112925724b0ded8e27779d741c86c62ecd1e: Status 404 returned error can't find the container with id 2e84f398f5ccdab653025f636579112925724b0ded8e27779d741c86c62ecd1e Jul 01 22:44:35.369624 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:35.369588 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" event={"ID":"a99dd773-8549-4e25-93a0-272c8b1cdbf2","Type":"ContainerStarted","Data":"239aa7b77d5764af43c2f29951f9e9d8c313846aa36ca1b1858e0f2915214504"} Jul 01 22:44:35.370075 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:35.369652 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" event={"ID":"a99dd773-8549-4e25-93a0-272c8b1cdbf2","Type":"ContainerStarted","Data":"2e84f398f5ccdab653025f636579112925724b0ded8e27779d741c86c62ecd1e"} Jul 01 22:44:35.391173 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:35.391123 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" podStartSLOduration=2.214628417 podStartE2EDuration="2.391104745s" totalImagesPullingTime="176.476328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:44:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:44:35.389260958 +0000 UTC m=+746.276518947" watchObservedRunningTime="2026-07-01 22:44:35.391104745 +0000 UTC m=+746.278362702" Jul 01 22:44:36.254064 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:36.254017 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:36.254277 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:36.254132 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:36.266771 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:36.266738 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:36.386424 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:36.386395 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls4cl" Jul 01 22:44:44.249704 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:44.249669 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:44.250160 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:44.249728 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:44.262180 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:44.262155 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:44.428602 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:44.428569 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5" Jul 01 22:44:51.129938 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.129891 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr"] Jul 01 22:44:51.137350 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.137311 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.140775 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.140729 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 01 22:44:51.142535 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.142491 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr"] Jul 01 22:44:51.205469 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.205395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.205758 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.205486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpq26\" (UniqueName: \"kubernetes.io/projected/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-kube-api-access-vpq26\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.205758 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.205581 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.205758 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.205663 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.205758 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.205725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.206000 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.205765 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307308 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307148 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307308 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307182 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307308 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307308 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307490 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpq26\" (UniqueName: \"kubernetes.io/projected/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-kube-api-access-vpq26\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307581 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307552 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307664 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307571 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.307664 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.307625 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.309604 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.309586 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.309713 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.309697 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.317548 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.317500 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpq26\" (UniqueName: \"kubernetes.io/projected/11fc4a09-f1a6-4c54-8fe0-964c55d0e122-kube-api-access-vpq26\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr\" (UID: \"11fc4a09-f1a6-4c54-8fe0-964c55d0e122\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.452168 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.452075 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:44:51.592123 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:51.592098 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr"] Jul 01 22:44:51.593698 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:44:51.593666 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11fc4a09_f1a6_4c54_8fe0_964c55d0e122.slice/crio-9ec74f265ee81c10669311138c1d0fae084fc433a1737babc33f26d1d3061a93 WatchSource:0}: Error finding container 9ec74f265ee81c10669311138c1d0fae084fc433a1737babc33f26d1d3061a93: Status 404 returned error can't find the container with id 9ec74f265ee81c10669311138c1d0fae084fc433a1737babc33f26d1d3061a93 Jul 01 22:44:52.458410 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:52.458373 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" event={"ID":"11fc4a09-f1a6-4c54-8fe0-964c55d0e122","Type":"ContainerStarted","Data":"560fe3a4819b92555c0a8a1c5b5cc0cbfb08fe0c060ca22dba025e9a51a2c32b"} Jul 01 22:44:52.458812 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:52.458420 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" event={"ID":"11fc4a09-f1a6-4c54-8fe0-964c55d0e122","Type":"ContainerStarted","Data":"9ec74f265ee81c10669311138c1d0fae084fc433a1737babc33f26d1d3061a93"} Jul 01 22:44:52.479572 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:44:52.479517 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" podStartSLOduration=1.293330345 podStartE2EDuration="1.479500767s" totalImagesPullingTime="186.170422ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:44:52.476245581 +0000 UTC m=+763.363503539" watchObservedRunningTime="2026-07-01 22:44:52.479500767 +0000 UTC m=+763.366758724" Jul 01 22:45:01.452984 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:45:01.452947 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:45:01.453448 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:45:01.453041 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:45:01.465510 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:45:01.465480 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:45:01.514831 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:45:01.514802 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr" Jul 01 22:45:59.795198 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:45:59.795162 2581 generic.go:356] "Generic (PLEG): container finished" podID="1413feed-61f7-49f4-92c1-2c413976b7c4" containerID="00d358406ef4aa15c4b98bedf60dc7adc041bb5a03a582a50a80cde3afc6cf09" exitCode=2 Jul 01 22:45:59.795590 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:45:59.795235 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" event={"ID":"1413feed-61f7-49f4-92c1-2c413976b7c4","Type":"ContainerDied","Data":"00d358406ef4aa15c4b98bedf60dc7adc041bb5a03a582a50a80cde3afc6cf09"} Jul 01 22:45:59.795590 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:45:59.795577 2581 scope.go:117] "RemoveContainer" containerID="00d358406ef4aa15c4b98bedf60dc7adc041bb5a03a582a50a80cde3afc6cf09" Jul 01 22:46:00.801864 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:00.801834 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" event={"ID":"1413feed-61f7-49f4-92c1-2c413976b7c4","Type":"ContainerStarted","Data":"4e31d0a3545a7debf54900b142794ff9ef4d196f059d91f3f626d8460f9e7563"} Jul 01 22:46:00.802314 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:00.802043 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:46:11.809488 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:11.809462 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:46:31.960599 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:31.960567 2581 generic.go:356] "Generic (PLEG): container finished" podID="1413feed-61f7-49f4-92c1-2c413976b7c4" containerID="4e31d0a3545a7debf54900b142794ff9ef4d196f059d91f3f626d8460f9e7563" exitCode=2 Jul 01 22:46:31.960932 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:31.960611 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" event={"ID":"1413feed-61f7-49f4-92c1-2c413976b7c4","Type":"ContainerDied","Data":"4e31d0a3545a7debf54900b142794ff9ef4d196f059d91f3f626d8460f9e7563"} Jul 01 22:46:31.960932 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:31.960660 2581 scope.go:117] "RemoveContainer" containerID="00d358406ef4aa15c4b98bedf60dc7adc041bb5a03a582a50a80cde3afc6cf09" Jul 01 22:46:31.960932 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:31.960920 2581 scope.go:117] "RemoveContainer" containerID="4e31d0a3545a7debf54900b142794ff9ef4d196f059d91f3f626d8460f9e7563" Jul 01 22:46:31.961199 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:46:31.961173 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7bbcb654f4-648v7_openshift-ingress(1413feed-61f7-49f4-92c1-2c413976b7c4)\"" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" podUID="1413feed-61f7-49f4-92c1-2c413976b7c4" Jul 01 22:46:32.040559 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:32.040531 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:46:32.967522 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:32.967492 2581 scope.go:117] "RemoveContainer" containerID="4e31d0a3545a7debf54900b142794ff9ef4d196f059d91f3f626d8460f9e7563" Jul 01 22:46:32.967891 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:46:32.967663 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7bbcb654f4-648v7_openshift-ingress(1413feed-61f7-49f4-92c1-2c413976b7c4)\"" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" podUID="1413feed-61f7-49f4-92c1-2c413976b7c4" Jul 01 22:46:37.030036 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:37.030005 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5bb9dd6b4f-6g5t6_1bb7843e-9d76-46de-a6ff-b726250400de/maas-api/0.log" Jul 01 22:46:37.252152 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:37.252118 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76c97789cb-c594c_36348a95-95fb-43ae-acd1-b4ba673a27a7/manager/0.log" Jul 01 22:46:37.610069 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:37.610022 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf_a911cef9-0bf8-40a5-94dc-279ab239559e/copy-manifests/0.log" Jul 01 22:46:37.617358 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:37.617337 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf_a911cef9-0bf8-40a5-94dc-279ab239559e/manager/0.log" Jul 01 22:46:40.458995 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:40.458967 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47q2rf_8ba2d047-39d0-4586-a0fe-d047e5df6dc6/istio-proxy/0.log" Jul 01 22:46:40.576224 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:40.576192 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b4d9t_9d33a8ff-9085-4ca8-8321-47e7670617c8/discovery/0.log" Jul 01 22:46:40.910196 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:40.910123 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8trpz_98ac7706-38c8-4b54-b2d3-fc58c7c6805a/istio-proxy/1.log" Jul 01 22:46:41.019447 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:41.019421 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-w822m_d4d2911b-02e5-4597-8b6c-5d3c19f8b148/payload-pre-processing/0.log" Jul 01 22:46:41.123921 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:41.123898 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-648v7_1413feed-61f7-49f4-92c1-2c413976b7c4/payload-processing/1.log" Jul 01 22:46:41.796113 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:41.796085 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ls4cl_69bcb5fd-1aca-40d2-9dff-738688081f81/main/0.log" Jul 01 22:46:41.807294 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:41.807270 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:46:41.807604 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:41.807590 2581 scope.go:117] "RemoveContainer" containerID="4e31d0a3545a7debf54900b142794ff9ef4d196f059d91f3f626d8460f9e7563" Jul 01 22:46:41.807785 ip-10-0-141-190 kubenswrapper[2581]: E0701 22:46:41.807768 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7bbcb654f4-648v7_openshift-ingress(1413feed-61f7-49f4-92c1-2c413976b7c4)\"" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" podUID="1413feed-61f7-49f4-92c1-2c413976b7c4" Jul 01 22:46:41.907003 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:41.906981 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbllsw5_a99dd773-8549-4e25-93a0-272c8b1cdbf2/main/0.log" Jul 01 22:46:42.118121 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:42.118035 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-46tpr_11fc4a09-f1a6-4c54-8fe0-964c55d0e122/main/0.log" Jul 01 22:46:49.160434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:49.160405 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4g9k9_889ac5e9-344c-4000-8257-66a0ef85e0eb/global-pull-secret-syncer/0.log" Jul 01 22:46:49.330058 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:49.330016 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lrs62_51564124-a668-464c-80ef-2985c3c6c109/konnectivity-agent/0.log" Jul 01 22:46:49.426836 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:49.426763 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-190.ec2.internal_1a76c7746494e50a2be891e8cc385fe8/haproxy/0.log" Jul 01 22:46:55.659455 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:55.659367 2581 scope.go:117] "RemoveContainer" containerID="4e31d0a3545a7debf54900b142794ff9ef4d196f059d91f3f626d8460f9e7563" Jul 01 22:46:56.004213 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.004187 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_01fbd166-f977-495e-bdfe-54d7bf955e10/alertmanager/0.log" Jul 01 22:46:56.026021 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.025982 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_01fbd166-f977-495e-bdfe-54d7bf955e10/config-reloader/0.log" Jul 01 22:46:56.047787 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.047759 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_01fbd166-f977-495e-bdfe-54d7bf955e10/kube-rbac-proxy-web/0.log" Jul 01 22:46:56.071558 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.071531 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_01fbd166-f977-495e-bdfe-54d7bf955e10/kube-rbac-proxy/0.log" Jul 01 22:46:56.087883 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.087854 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" event={"ID":"1413feed-61f7-49f4-92c1-2c413976b7c4","Type":"ContainerStarted","Data":"755cfb30a4417543c611bdf26014e1b25421e93d6bfa91387120ef57db2c5a97"} Jul 01 22:46:56.088112 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.088093 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:46:56.093564 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.093545 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_01fbd166-f977-495e-bdfe-54d7bf955e10/kube-rbac-proxy-metric/0.log" Jul 01 22:46:56.121451 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.121421 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_01fbd166-f977-495e-bdfe-54d7bf955e10/prom-label-proxy/0.log" Jul 01 22:46:56.142573 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.142550 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_01fbd166-f977-495e-bdfe-54d7bf955e10/init-config-reloader/0.log" Jul 01 22:46:56.216679 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.216644 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-bxhr5_1595ad19-6cd6-423a-8753-b2346148bf55/kube-state-metrics/0.log" Jul 01 22:46:56.237434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.237396 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-bxhr5_1595ad19-6cd6-423a-8753-b2346148bf55/kube-rbac-proxy-main/0.log" Jul 01 22:46:56.262752 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.262670 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-bxhr5_1595ad19-6cd6-423a-8753-b2346148bf55/kube-rbac-proxy-self/0.log" Jul 01 22:46:56.292007 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.291976 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-bd5b6998f-wrwcv_f6be1d7e-16b9-4a73-8149-309a42dbebb9/metrics-server/0.log" Jul 01 22:46:56.317595 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.317571 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-fw6cl_7c63d7a1-9f56-48da-8282-f2d3d6894b42/monitoring-plugin/0.log" Jul 01 22:46:56.357134 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.357105 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6rz7c_42677453-af61-4eca-8b32-38c8c59f455a/node-exporter/0.log" Jul 01 22:46:56.379640 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.379607 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6rz7c_42677453-af61-4eca-8b32-38c8c59f455a/kube-rbac-proxy/0.log" Jul 01 22:46:56.407099 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.407072 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6rz7c_42677453-af61-4eca-8b32-38c8c59f455a/init-textfile/0.log" Jul 01 22:46:56.677027 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.676994 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-l6hc2_f6d438d2-58b4-445e-aac3-b08dc12f116c/kube-rbac-proxy-main/0.log" Jul 01 22:46:56.701794 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.701762 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-l6hc2_f6d438d2-58b4-445e-aac3-b08dc12f116c/kube-rbac-proxy-self/0.log" Jul 01 22:46:56.724526 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.724498 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-l6hc2_f6d438d2-58b4-445e-aac3-b08dc12f116c/openshift-state-metrics/0.log" Jul 01 22:46:56.780935 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.780851 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c055538-5eb9-499a-8c2c-dc5ab88bb47d/prometheus/0.log" Jul 01 22:46:56.803034 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.803003 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c055538-5eb9-499a-8c2c-dc5ab88bb47d/config-reloader/0.log" Jul 01 22:46:56.823801 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.823772 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c055538-5eb9-499a-8c2c-dc5ab88bb47d/thanos-sidecar/0.log" Jul 01 22:46:56.849389 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.849361 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c055538-5eb9-499a-8c2c-dc5ab88bb47d/kube-rbac-proxy-web/0.log" Jul 01 22:46:56.870933 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.870908 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c055538-5eb9-499a-8c2c-dc5ab88bb47d/kube-rbac-proxy/0.log" Jul 01 22:46:56.893505 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.893480 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c055538-5eb9-499a-8c2c-dc5ab88bb47d/kube-rbac-proxy-thanos/0.log" Jul 01 22:46:56.915113 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.915080 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c055538-5eb9-499a-8c2c-dc5ab88bb47d/init-config-reloader/0.log" Jul 01 22:46:56.946503 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.946473 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-fkz95_f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06/prometheus-operator/0.log" Jul 01 22:46:56.967075 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:56.967028 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-fkz95_f6386ca7-3683-4f9a-bb84-7b0bc0eb8d06/kube-rbac-proxy/0.log" Jul 01 22:46:57.025639 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.025613 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85ddcb5c4b-bhvwx_42d35cee-db37-40ff-8678-f91ae118c35c/telemeter-client/0.log" Jul 01 22:46:57.052868 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.052793 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85ddcb5c4b-bhvwx_42d35cee-db37-40ff-8678-f91ae118c35c/reload/0.log" Jul 01 22:46:57.074434 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.074405 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85ddcb5c4b-bhvwx_42d35cee-db37-40ff-8678-f91ae118c35c/kube-rbac-proxy/0.log" Jul 01 22:46:57.112106 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.112077 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd5bf976-qwkx9_2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2/thanos-query/0.log" Jul 01 22:46:57.133385 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.133356 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd5bf976-qwkx9_2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2/kube-rbac-proxy-web/0.log" Jul 01 22:46:57.157840 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.157812 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd5bf976-qwkx9_2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2/kube-rbac-proxy/0.log" Jul 01 22:46:57.177784 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.177756 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd5bf976-qwkx9_2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2/prom-label-proxy/0.log" Jul 01 22:46:57.198720 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.198694 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd5bf976-qwkx9_2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2/kube-rbac-proxy-rules/0.log" Jul 01 22:46:57.219228 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:57.219201 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd5bf976-qwkx9_2ba3edfb-832f-4e7d-bc1a-f1c4406cbfe2/kube-rbac-proxy-metrics/0.log" Jul 01 22:46:59.162270 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.162222 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8"] Jul 01 22:46:59.166305 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.166282 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.168880 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.168854 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c2hxn\"/\"kube-root-ca.crt\"" Jul 01 22:46:59.169016 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.168989 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c2hxn\"/\"openshift-service-ca.crt\"" Jul 01 22:46:59.170235 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.170207 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c2hxn\"/\"default-dockercfg-l5t7j\"" Jul 01 22:46:59.176168 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.176138 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8"] Jul 01 22:46:59.273109 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.273042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-proc\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.273109 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.273113 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4sqt\" (UniqueName: \"kubernetes.io/projected/01d8c004-bb93-40df-8cf8-102dbd64058e-kube-api-access-d4sqt\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.273354 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.273176 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-podres\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.273354 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.273213 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-lib-modules\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.273354 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.273253 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-sys\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.374597 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374565 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-proc\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.374597 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4sqt\" (UniqueName: \"kubernetes.io/projected/01d8c004-bb93-40df-8cf8-102dbd64058e-kube-api-access-d4sqt\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.374851 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374720 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-proc\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.374851 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-podres\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.374851 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374756 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-lib-modules\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.374851 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374804 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-sys\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.375124 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374875 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-podres\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.375124 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374942 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-lib-modules\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.375124 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.374963 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01d8c004-bb93-40df-8cf8-102dbd64058e-sys\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.383214 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.383192 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4sqt\" (UniqueName: \"kubernetes.io/projected/01d8c004-bb93-40df-8cf8-102dbd64058e-kube-api-access-d4sqt\") pod \"perf-node-gather-daemonset-t6xr8\" (UID: \"01d8c004-bb93-40df-8cf8-102dbd64058e\") " pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.478383 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.478353 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:46:59.616038 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:46:59.616011 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8"] Jul 01 22:46:59.617712 ip-10-0-141-190 kubenswrapper[2581]: W0701 22:46:59.617676 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod01d8c004_bb93_40df_8cf8_102dbd64058e.slice/crio-62a2b02aff307b9c77c1b8f55b8f7f011c237acb0ea41ec13540a96a63b190a1 WatchSource:0}: Error finding container 62a2b02aff307b9c77c1b8f55b8f7f011c237acb0ea41ec13540a96a63b190a1: Status 404 returned error can't find the container with id 62a2b02aff307b9c77c1b8f55b8f7f011c237acb0ea41ec13540a96a63b190a1 Jul 01 22:47:00.110244 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:00.110139 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" event={"ID":"01d8c004-bb93-40df-8cf8-102dbd64058e","Type":"ContainerStarted","Data":"e7f7705d93e38a813017e639b59eba6fdd6312a4bde6a9dc96c534f699be441a"} Jul 01 22:47:00.110244 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:00.110190 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" event={"ID":"01d8c004-bb93-40df-8cf8-102dbd64058e","Type":"ContainerStarted","Data":"62a2b02aff307b9c77c1b8f55b8f7f011c237acb0ea41ec13540a96a63b190a1"} Jul 01 22:47:00.110444 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:00.110248 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:47:00.126676 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:00.126618 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" podStartSLOduration=1.126604044 podStartE2EDuration="1.126604044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:47:00.125254156 +0000 UTC m=+891.012512123" watchObservedRunningTime="2026-07-01 22:47:00.126604044 +0000 UTC m=+891.013862000" Jul 01 22:47:01.348730 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:01.348656 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rwvtw_4eaf10c2-65d3-44bb-a063-2011878aba2b/dns/0.log" Jul 01 22:47:01.372007 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:01.371983 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rwvtw_4eaf10c2-65d3-44bb-a063-2011878aba2b/kube-rbac-proxy/0.log" Jul 01 22:47:01.515451 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:01.515418 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c92kx_ba4ff25c-4df9-47ff-8c80-5e1a10867213/dns-node-resolver/0.log" Jul 01 22:47:02.082472 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:02.082443 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pxjt_09e5d35c-7bb6-4eda-b264-e25731987f58/node-ca/0.log" Jul 01 22:47:03.034377 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:03.034347 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47q2rf_8ba2d047-39d0-4586-a0fe-d047e5df6dc6/istio-proxy/0.log" Jul 01 22:47:03.077186 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:03.077159 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b4d9t_9d33a8ff-9085-4ca8-8321-47e7670617c8/discovery/0.log" Jul 01 22:47:03.156728 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:03.156699 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8trpz_98ac7706-38c8-4b54-b2d3-fc58c7c6805a/istio-proxy/1.log" Jul 01 22:47:03.156911 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:03.156898 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8trpz_98ac7706-38c8-4b54-b2d3-fc58c7c6805a/istio-proxy/0.log" Jul 01 22:47:03.176506 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:03.176484 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-w822m_d4d2911b-02e5-4597-8b6c-5d3c19f8b148/payload-pre-processing/0.log" Jul 01 22:47:03.201015 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:03.200990 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-648v7_1413feed-61f7-49f4-92c1-2c413976b7c4/payload-processing/2.log" Jul 01 22:47:03.201619 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:03.201603 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-648v7_1413feed-61f7-49f4-92c1-2c413976b7c4/payload-processing/1.log" Jul 01 22:47:03.905955 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:03.905927 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xm7xn_6aa2539a-b2b3-4e99-a5d2-13c56f919590/serve-healthcheck-canary/0.log" Jul 01 22:47:04.404333 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:04.404293 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4f4kd_1ae01200-de56-4195-a919-151c0a717eed/kube-rbac-proxy/0.log" Jul 01 22:47:04.429551 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:04.429529 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4f4kd_1ae01200-de56-4195-a919-151c0a717eed/exporter/0.log" Jul 01 22:47:04.455957 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:04.455930 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4f4kd_1ae01200-de56-4195-a919-151c0a717eed/extractor/0.log" Jul 01 22:47:06.125398 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:06.125373 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c2hxn/perf-node-gather-daemonset-t6xr8" Jul 01 22:47:06.929706 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:06.929678 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5bb9dd6b4f-6g5t6_1bb7843e-9d76-46de-a6ff-b726250400de/maas-api/0.log" Jul 01 22:47:07.009857 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:07.009834 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76c97789cb-c594c_36348a95-95fb-43ae-acd1-b4ba673a27a7/manager/0.log" Jul 01 22:47:07.094913 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:07.094887 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-648v7" Jul 01 22:47:07.129504 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:07.129479 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf_a911cef9-0bf8-40a5-94dc-279ab239559e/manager/0.log" Jul 01 22:47:07.158533 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:07.158452 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cf7bccf5c-g8qmf_a911cef9-0bf8-40a5-94dc-279ab239559e/copy-manifests/0.log" Jul 01 22:47:08.847555 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:08.847522 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d78589cfd-k6777_ba660d3b-e069-4fa4-a480-8cf55a4122df/manager/0.log" Jul 01 22:47:08.874605 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:08.874580 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9hvrn_2fb8a89f-ce78-43b4-b27a-176ab1e7a5a9/openshift-lws-operator/0.log" Jul 01 22:47:09.935527 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:09.935446 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47q2rf_8ba2d047-39d0-4586-a0fe-d047e5df6dc6/istio-proxy/0.log" Jul 01 22:47:09.973969 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:09.973946 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b4d9t_9d33a8ff-9085-4ca8-8321-47e7670617c8/discovery/0.log" Jul 01 22:47:10.023395 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:10.023351 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8trpz_98ac7706-38c8-4b54-b2d3-fc58c7c6805a/istio-proxy/1.log" Jul 01 22:47:10.023632 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:10.023616 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8trpz_98ac7706-38c8-4b54-b2d3-fc58c7c6805a/istio-proxy/0.log" Jul 01 22:47:10.043201 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:10.043163 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-w822m_d4d2911b-02e5-4597-8b6c-5d3c19f8b148/payload-pre-processing/0.log" Jul 01 22:47:10.056677 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:10.056656 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-648v7_1413feed-61f7-49f4-92c1-2c413976b7c4/payload-processing/2.log" Jul 01 22:47:10.057227 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:10.057212 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-648v7_1413feed-61f7-49f4-92c1-2c413976b7c4/payload-processing/1.log" Jul 01 22:47:11.165919 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:11.165864 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47q2rf_8ba2d047-39d0-4586-a0fe-d047e5df6dc6/istio-proxy/0.log" Jul 01 22:47:11.206837 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:11.206802 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b4d9t_9d33a8ff-9085-4ca8-8321-47e7670617c8/discovery/0.log" Jul 01 22:47:11.254662 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:11.254636 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8trpz_98ac7706-38c8-4b54-b2d3-fc58c7c6805a/istio-proxy/1.log" Jul 01 22:47:11.254763 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:11.254746 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8trpz_98ac7706-38c8-4b54-b2d3-fc58c7c6805a/istio-proxy/0.log" Jul 01 22:47:11.266008 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:11.265989 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-w822m_d4d2911b-02e5-4597-8b6c-5d3c19f8b148/payload-pre-processing/0.log" Jul 01 22:47:11.281661 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:11.281639 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-648v7_1413feed-61f7-49f4-92c1-2c413976b7c4/payload-processing/2.log" Jul 01 22:47:11.282144 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:11.282130 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-648v7_1413feed-61f7-49f4-92c1-2c413976b7c4/payload-processing/1.log" Jul 01 22:47:15.430894 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:15.430862 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhm8h_31506d3f-0743-4684-8b09-aa952f816530/kube-multus-additional-cni-plugins/0.log" Jul 01 22:47:15.452365 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:15.452341 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhm8h_31506d3f-0743-4684-8b09-aa952f816530/egress-router-binary-copy/0.log" Jul 01 22:47:15.472607 ip-10-0-141-190 kubenswrapper[2581]: I0701 22:47:15.472579 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhm8h_31506d3f-0743-4684-8b09-aa952f816530/cni-plugins/0.log"