Jul 07 18:44:56.408134 ip-10-0-141-163 systemd[1]: Starting Kubernetes Kubelet... Jul 07 18:44:56.817448 ip-10-0-141-163 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:44:56.817448 ip-10-0-141-163 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 18:44:56.817448 ip-10-0-141-163 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:44:56.817448 ip-10-0-141-163 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 18:44:56.817448 ip-10-0-141-163 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:44:56.818171 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.817433 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 18:44:56.826009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.825979 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:44:56.826009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826004 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:44:56.826009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826009 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:44:56.826009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826012 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:44:56.826009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826018 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826021 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826025 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826028 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826030 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826034 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826036 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826039 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826042 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826045 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826048 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826050 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826053 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826055 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826058 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826060 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826063 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826065 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826068 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:44:56.826212 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826071 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826073 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826076 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826079 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826081 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826084 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826086 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826089 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826095 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826098 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826101 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826103 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826106 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826111 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826115 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826118 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826121 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826124 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826127 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:44:56.826660 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826130 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826133 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826136 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826139 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826141 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826144 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826146 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826149 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826151 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826153 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826156 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826158 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826161 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826165 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826170 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826174 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826177 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826180 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826182 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826185 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:44:56.827160 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826188 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826191 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826193 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826196 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826198 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826201 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826203 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826206 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826209 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826212 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826214 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826217 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826220 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826222 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826225 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826229 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826232 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826234 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826237 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826239 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:44:56.827637 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826242 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826245 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826248 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826251 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826253 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826257 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826259 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826263 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826268 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826270 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826273 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826276 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826280 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826283 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826286 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826288 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.826291 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827034 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827043 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827046 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:44:56.828141 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827049 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827052 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827055 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827058 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827061 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827064 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827068 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827072 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827075 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827078 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827081 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827083 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827086 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827089 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827092 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827094 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827097 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827100 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827102 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:44:56.828621 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827105 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827107 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827110 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827113 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827115 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827118 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827121 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827123 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827125 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827128 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827131 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827134 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827136 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827139 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827141 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827144 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827146 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827149 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827151 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827155 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:44:56.829101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827157 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827160 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827162 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827166 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827168 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827171 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827173 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827176 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827179 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827183 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827187 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827190 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827193 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827195 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827199 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827202 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827205 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827207 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827210 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:44:56.829579 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827213 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827215 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827218 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827220 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827223 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827226 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827229 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827232 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827234 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827237 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827239 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827241 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827244 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827247 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827250 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827253 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827255 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827258 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827260 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827263 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:44:56.830056 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827265 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827267 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827270 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827274 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827277 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827280 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827283 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827286 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827289 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827292 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827294 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827297 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827299 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827302 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827304 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827307 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827309 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827311 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827692 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:44:56.830549 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827696 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827699 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827702 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827705 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827708 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827711 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827713 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827716 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827719 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827722 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827724 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827727 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827729 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827732 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827734 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827737 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827740 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827742 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827745 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827747 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:44:56.831022 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827750 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827753 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827756 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827758 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827761 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827763 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827766 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827768 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827771 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827773 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827776 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827778 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827781 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827785 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827789 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827792 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827796 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827799 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827802 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827805 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:44:56.831493 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827807 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827810 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827812 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827814 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827817 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827834 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827837 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827839 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827842 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827845 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827848 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827850 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827853 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827855 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827858 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827861 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827863 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827866 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827869 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827871 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:44:56.832009 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827874 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827877 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827879 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827882 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827884 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827886 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827889 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827891 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827894 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827896 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827899 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827901 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827904 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827906 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827909 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827918 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827924 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827927 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827929 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:44:56.832487 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827932 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827934 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827938 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827941 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827944 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827947 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827949 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827952 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827954 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827958 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827961 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827964 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827966 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827969 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827971 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827974 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827976 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827978 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.827981 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828055 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 18:44:56.832973 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828062 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828070 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828077 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828083 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828088 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828094 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828099 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828102 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828105 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828108 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828111 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828114 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828117 2582 flags.go:64] FLAG: --cgroup-root="" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828119 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828122 2582 flags.go:64] FLAG: --client-ca-file="" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828125 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828129 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828133 2582 flags.go:64] FLAG: --cluster-domain="" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828136 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828139 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828142 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828145 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828149 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 18:44:56.833449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828152 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828158 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828161 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828165 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828168 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828170 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828173 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828176 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828180 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828183 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828186 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828188 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828191 2582 flags.go:64] FLAG: --enable-server="true" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828194 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828198 2582 flags.go:64] FLAG: --event-burst="100" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828201 2582 flags.go:64] FLAG: --event-qps="50" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828203 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828207 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828210 2582 flags.go:64] FLAG: --eviction-hard="" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828218 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828220 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828223 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828226 2582 flags.go:64] FLAG: --eviction-soft="" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828229 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828231 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 18:44:56.834012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828235 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828238 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828241 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828244 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828246 2582 flags.go:64] FLAG: --feature-gates="" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828250 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828253 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828256 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828260 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828263 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828266 2582 flags.go:64] FLAG: --help="false" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828269 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-141-163.ec2.internal" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828272 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828275 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828277 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828281 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828284 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828287 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828289 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828292 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828295 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828298 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828300 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828303 2582 flags.go:64] FLAG: --kube-reserved="" Jul 07 18:44:56.834639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828306 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828309 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828311 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828314 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828317 2582 flags.go:64] FLAG: --lock-file="" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828319 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828322 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828325 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828330 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828333 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828336 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828339 2582 flags.go:64] FLAG: --logging-format="text" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828341 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828344 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828347 2582 flags.go:64] FLAG: --manifest-url="" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828349 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828355 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828358 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828362 2582 flags.go:64] FLAG: --max-pods="110" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828365 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828368 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828370 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828373 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828376 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828378 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 18:44:56.835292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828381 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828389 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828392 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828395 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828397 2582 flags.go:64] FLAG: --pod-cidr="" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828400 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828405 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828408 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828411 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828413 2582 flags.go:64] FLAG: --port="10250" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828416 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828419 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-080fb86ef1e4d5bde" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828422 2582 flags.go:64] FLAG: --qos-reserved="" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828425 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828427 2582 flags.go:64] FLAG: --register-node="true" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828430 2582 flags.go:64] FLAG: --register-with-taints="" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828435 2582 flags.go:64] FLAG: --registry-burst="10" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828439 2582 flags.go:64] FLAG: --registry-qps="5" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828442 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828444 2582 flags.go:64] FLAG: --reserved-memory="" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828448 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828451 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828454 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828458 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828461 2582 flags.go:64] FLAG: --runonce="false" Jul 07 18:44:56.835929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828463 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828467 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828469 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828472 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828475 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828478 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828481 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828483 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828486 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828489 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828491 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828494 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828497 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828499 2582 flags.go:64] FLAG: --system-cgroups="" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828502 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828507 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828509 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828512 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828517 2582 flags.go:64] FLAG: --tls-min-version="" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828519 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828522 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828524 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828527 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828531 2582 flags.go:64] FLAG: --v="2" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828535 2582 flags.go:64] FLAG: --version="false" Jul 07 18:44:56.836534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828539 2582 flags.go:64] FLAG: --vmodule="" Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828543 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828546 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828678 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828682 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828686 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828689 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828692 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828695 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828698 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828700 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828703 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828705 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828708 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828710 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828714 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828717 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828720 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828723 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:44:56.837143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828726 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828728 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828731 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828734 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828737 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828739 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828741 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828744 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828746 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828749 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828751 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828756 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828758 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828761 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828764 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828767 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828769 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828772 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828775 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828777 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:44:56.837641 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828780 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828782 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828784 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828787 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828789 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828792 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828794 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828797 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828799 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828802 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828804 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828807 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828809 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828812 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828814 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828816 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828833 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828836 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828838 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828841 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828844 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:44:56.838138 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828846 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828849 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828853 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828855 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828858 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828860 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828863 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828866 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828868 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828872 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828875 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828877 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828880 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828882 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828886 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828890 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828893 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828896 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828900 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:44:56.838639 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828903 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828905 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828908 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828910 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828913 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828915 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828918 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828921 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828924 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828926 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828940 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828943 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828947 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828949 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828952 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828956 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828959 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828962 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828964 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:44:56.839214 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828967 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828970 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828972 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.828976 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.828984 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.836240 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.836256 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.836276 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.836282 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836335 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836339 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836343 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836346 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836349 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836351 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:44:56.839682 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836353 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836356 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836359 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836362 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836364 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836367 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836369 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836372 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836374 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836377 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836379 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836382 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836384 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836386 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836390 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836392 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836395 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836397 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836400 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836402 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:44:56.840075 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836405 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836409 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836413 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836416 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836419 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836422 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836424 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836427 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836430 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836432 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836435 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836437 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836440 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836442 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836445 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836447 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836449 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836452 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836454 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:44:56.840550 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836457 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836460 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836462 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836465 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836467 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836470 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836472 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836475 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836478 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836480 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836482 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836485 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836488 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836490 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836493 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836496 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836498 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836501 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836504 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836506 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:44:56.841213 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836509 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836512 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836514 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836516 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836519 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836521 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836523 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836526 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836528 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836530 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836533 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836536 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836540 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836543 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836545 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836548 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836551 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836553 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836557 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:44:56.841686 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836561 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836564 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836567 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836570 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836573 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836575 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836578 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836581 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836584 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836586 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836589 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836592 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836595 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836597 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836600 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:44:56.842156 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.836604 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836740 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836745 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836749 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836751 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836754 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836757 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836761 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836764 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836768 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836770 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836773 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836776 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836778 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836781 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836784 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836786 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836788 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836791 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:44:56.842530 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836793 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836795 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836798 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836800 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836803 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836805 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836807 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836810 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836812 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836814 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836817 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836837 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836839 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836842 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836845 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836847 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836850 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836852 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836855 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836857 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:44:56.843002 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836860 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836862 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836865 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836867 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836870 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836872 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836875 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836877 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836880 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836882 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836885 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836887 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836889 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836892 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836894 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836896 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836899 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836901 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836904 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836906 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:44:56.843497 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836908 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836910 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836914 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836919 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836921 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836924 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836927 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836929 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836932 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836935 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836938 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836941 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836944 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836947 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836949 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836952 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836954 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836957 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836959 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:44:56.844040 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836962 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836964 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836967 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836969 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836972 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836974 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836976 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836979 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836981 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836983 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836986 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836988 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836990 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836993 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836996 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.836998 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.837001 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.837003 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.837006 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.837008 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:44:56.844492 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.837010 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:44:56.845317 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:56.837013 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:44:56.845317 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.837017 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:44:56.845317 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.837135 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 18:44:56.845317 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.839222 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 18:44:56.845317 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.840755 2582 server.go:1019] "Starting client certificate rotation" Jul 07 18:44:56.845317 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.840858 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 18:44:56.845317 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.840885 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 18:44:56.845317 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.843343 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 18:44:56.845785 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.845764 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 18:44:56.857740 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.857723 2582 log.go:25] "Validated CRI v1 runtime API" Jul 07 18:44:56.859737 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.859723 2582 log.go:25] "Validated CRI v1 image API" Jul 07 18:44:56.861339 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.861321 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 18:44:56.864410 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.864381 2582 fs.go:135] Filesystem UUIDs: map[6b405ec3-e685-4980-ac65-994a47ee972d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 91ba515e-5c82-4b28-b0a1-313773e7f597:/dev/nvme0n1p3] Jul 07 18:44:56.864484 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.864408 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 18:44:56.867976 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.867956 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 18:44:56.869625 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.869494 2582 manager.go:216] Machine: {Timestamp:2026-07-07 18:44:56.868086929 +0000 UTC m=+0.346027563 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3116157 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec201342b51d5e8e286fe3dbdaa88bde SystemUUID:ec201342-b51d-5e8e-286f-e3dbdaa88bde BootID:6281e17a-3430-4eef-ba17-d00733f37873 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3b:1a:e1:37:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3b:1a:e1:37:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:fb:0c:fe:6a:05 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 18:44:56.869625 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.869619 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 18:44:56.869774 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.869724 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 18:44:56.872357 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.872328 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 18:44:56.872517 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.872359 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-163.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 18:44:56.873641 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.873628 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 18:44:56.873705 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.873647 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 18:44:56.873705 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.873657 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 18:44:56.873705 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.873675 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 18:44:56.874986 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.874973 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 18:44:56.878163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878147 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 07 18:44:56.878280 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878271 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 18:44:56.878310 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878302 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 07 18:44:56.878344 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878312 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 18:44:56.878344 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878328 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 18:44:56.878344 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878339 2582 kubelet.go:398] "Adding apiserver pod source" Jul 07 18:44:56.878443 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878347 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 18:44:56.878800 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878788 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 18:44:56.878862 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.878810 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 18:44:56.880970 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.880951 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 18:44:56.883042 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.883029 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 18:44:56.883100 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.883059 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 18:44:56.884118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884101 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 18:44:56.884118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884121 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884131 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884139 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884151 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884163 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884174 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884181 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884190 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884198 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884209 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 18:44:56.884267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.884217 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 18:44:56.885771 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.885758 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 18:44:56.885844 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.885774 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 18:44:56.889132 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.889118 2582 server.go:1268] "Started kubelet" Jul 07 18:44:56.889246 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.889215 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 18:44:56.889344 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.889292 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 18:44:56.889389 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.889369 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 18:44:56.889853 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.889807 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 18:44:56.889931 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.889866 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-163.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 18:44:56.889995 ip-10-0-141-163 systemd[1]: Started Kubernetes Kubelet. Jul 07 18:44:56.891123 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.890304 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 18:44:56.891598 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.891577 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 18:44:56.892841 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.892802 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 07 18:44:56.895059 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.895042 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 18:44:56.897100 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.897084 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wjbcc" Jul 07 18:44:56.899029 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.899018 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 18:44:56.899115 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.899094 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:56.899172 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.899123 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 18:44:56.899700 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.899655 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 18:44:56.899700 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.899673 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 18:44:56.899700 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.899690 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 18:44:56.899875 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.899771 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 07 18:44:56.899875 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.899796 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 07 18:44:56.899984 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.898705 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-163.ec2.internal.18c0163b6ac80f3c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-163.ec2.internal,UID:ip-10-0-141-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-163.ec2.internal,},FirstTimestamp:2026-07-07 18:44:56.889093948 +0000 UTC m=+0.367034634,LastTimestamp:2026-07-07 18:44:56.889093948 +0000 UTC m=+0.367034634,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-163.ec2.internal,}" Jul 07 18:44:56.900326 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.900309 2582 factory.go:55] Registering systemd factory Jul 07 18:44:56.900326 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.900327 2582 factory.go:223] Registration of the systemd container factory successfully Jul 07 18:44:56.901267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.900568 2582 factory.go:153] Registering CRI-O factory Jul 07 18:44:56.901267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.900583 2582 factory.go:223] Registration of the crio container factory successfully Jul 07 18:44:56.901267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.900630 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 18:44:56.901267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.900656 2582 factory.go:103] Registering Raw factory Jul 07 18:44:56.901267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.900674 2582 manager.go:1196] Started watching for new ooms in manager Jul 07 18:44:56.901267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.901233 2582 manager.go:349] Starting recovery of all containers Jul 07 18:44:56.904750 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.904721 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 18:44:56.905205 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.905177 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 18:44:56.911759 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.911534 2582 manager.go:354] Recovery completed Jul 07 18:44:56.912104 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.912085 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wjbcc" Jul 07 18:44:56.916327 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.916314 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:44:56.918102 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.918087 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:44:56.918150 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.918116 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:44:56.918150 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.918126 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:44:56.918616 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.918604 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 18:44:56.918683 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.918616 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 18:44:56.918683 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.918630 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 07 18:44:56.918683 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.918676 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:44:56.919315 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.919298 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:44:56.919385 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.919327 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:44:56.919385 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.919337 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:44:56.920112 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.920095 2582 policy_none.go:49] "None policy: Start" Jul 07 18:44:56.920112 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.920114 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 18:44:56.920234 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.920130 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 18:44:56.920352 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.920281 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-163.ec2.internal.18c0163b6c82b0e6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-163.ec2.internal,UID:ip-10-0-141-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-163.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-163.ec2.internal,},FirstTimestamp:2026-07-07 18:44:56.918102246 +0000 UTC m=+0.396042879,LastTimestamp:2026-07-07 18:44:56.918102246 +0000 UTC m=+0.396042879,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-163.ec2.internal,}" Jul 07 18:44:56.921492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.921474 2582 policy_none.go:47] "Start" Jul 07 18:44:56.962399 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.962382 2582 manager.go:337] "Starting Device Plugin manager" Jul 07 18:44:56.962498 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.962428 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 18:44:56.962498 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.962442 2582 server.go:87] "Starting device plugin registration server" Jul 07 18:44:56.962699 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.962686 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 18:44:56.962771 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.962703 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 18:44:56.962842 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.962789 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 18:44:56.963009 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.962995 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 18:44:56.963009 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:56.963010 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 18:44:56.963737 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.963721 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 18:44:56.963860 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:56.963765 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.063049 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.063021 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:44:57.064765 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.064751 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:44:57.064851 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.064782 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:44:57.064851 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.064797 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:44:57.064851 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.064848 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.076591 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.076549 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.076591 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.076571 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-163.ec2.internal\": node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.115901 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.115877 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.124283 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.124255 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 18:44:57.125584 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.125566 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 18:44:57.125652 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.125604 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 18:44:57.125652 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.125625 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 18:44:57.125726 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.125658 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 18:44:57.130349 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.130334 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:44:57.216857 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.216811 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.225983 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.225955 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal"] Jul 07 18:44:57.226041 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.226024 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:44:57.227458 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.227444 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:44:57.227534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.227473 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:44:57.227534 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.227483 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:44:57.228786 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.228774 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:44:57.228914 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.228901 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.228958 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.228930 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:44:57.229570 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.229553 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:44:57.229624 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.229582 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:44:57.229624 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.229595 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:44:57.229995 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.229979 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:44:57.230051 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.230011 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:44:57.230051 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.230026 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:44:57.231169 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.231156 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.231218 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.231182 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:44:57.231854 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.231839 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:44:57.231931 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.231867 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:44:57.231931 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.231881 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:44:57.260436 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.260416 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-163.ec2.internal\" not found" node="ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.264975 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.264962 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-163.ec2.internal\" not found" node="ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.301980 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.301961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b50e609bf668c243e15e77d050b97b9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-163.ec2.internal\" (UID: \"5b50e609bf668c243e15e77d050b97b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.302050 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.301984 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d15639f345b044a83529457ff7d48d9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal\" (UID: \"1d15639f345b044a83529457ff7d48d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.302050 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.302003 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d15639f345b044a83529457ff7d48d9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal\" (UID: \"1d15639f345b044a83529457ff7d48d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.317277 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.317260 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.403192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.403118 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b50e609bf668c243e15e77d050b97b9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-163.ec2.internal\" (UID: \"5b50e609bf668c243e15e77d050b97b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.403192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.403140 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b50e609bf668c243e15e77d050b97b9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-163.ec2.internal\" (UID: \"5b50e609bf668c243e15e77d050b97b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.403192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.403169 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d15639f345b044a83529457ff7d48d9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal\" (UID: \"1d15639f345b044a83529457ff7d48d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.403383 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.403194 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d15639f345b044a83529457ff7d48d9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal\" (UID: \"1d15639f345b044a83529457ff7d48d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.403383 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.403229 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1d15639f345b044a83529457ff7d48d9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal\" (UID: \"1d15639f345b044a83529457ff7d48d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.403383 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.403235 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d15639f345b044a83529457ff7d48d9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal\" (UID: \"1d15639f345b044a83529457ff7d48d9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.418236 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.418204 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.518836 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.518781 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.563982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.563960 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.568642 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.568624 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:57.619636 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.619597 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.720161 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.720097 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.820700 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.820667 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:57.839855 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.839809 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 18:44:57.840007 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.839989 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 18:44:57.914320 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.914286 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 18:39:58 +0000 UTC" deadline="2027-12-10 00:39:51.626817189 +0000 UTC" Jul 07 18:44:57.914320 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:57.914312 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12485h54m53.712508149s" Jul 07 18:44:57.921442 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:57.921422 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:58.022144 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:58.022068 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:59.555774 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:59.555753 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:44:59.575977 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:59.575948 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:59.595276 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:59.595256 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:44:59.635661 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:59.635610 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b50e609bf668c243e15e77d050b97b9.slice/crio-b8de1c9c156684fa1cf5a33a2e1c856fa4709fdf3d14c4a0b57413acee4f6367 WatchSource:0}: Error finding container b8de1c9c156684fa1cf5a33a2e1c856fa4709fdf3d14c4a0b57413acee4f6367: Status 404 returned error can't find the container with id b8de1c9c156684fa1cf5a33a2e1c856fa4709fdf3d14c4a0b57413acee4f6367 Jul 07 18:44:59.636908 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:44:59.636884 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d15639f345b044a83529457ff7d48d9.slice/crio-a1ca130896024ec08ccd8812219a22a1cc9653b1e5aaaa5424e097211287288a WatchSource:0}: Error finding container a1ca130896024ec08ccd8812219a22a1cc9653b1e5aaaa5424e097211287288a: Status 404 returned error can't find the container with id a1ca130896024ec08ccd8812219a22a1cc9653b1e5aaaa5424e097211287288a Jul 07 18:44:59.640052 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:59.640030 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:44:59.677003 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:59.676975 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:59.777557 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:44:59.777492 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-163.ec2.internal\" not found" Jul 07 18:44:59.827780 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:59.827336 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:44:59.852353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:59.852327 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" Jul 07 18:44:59.861443 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:59.861424 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 18:44:59.862354 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:59.862342 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" Jul 07 18:44:59.874241 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:44:59.874223 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 18:45:00.332314 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.332285 2582 apiserver.go:52] "Watching apiserver" Jul 07 18:45:00.338692 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.338472 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 18:45:00.339017 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.338932 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-b4hlp","openshift-cluster-node-tuning-operator/tuned-w2bwn","openshift-dns/node-resolver-5wwvt","openshift-image-registry/node-ca-f8lmc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal","openshift-multus/multus-dsl26","openshift-multus/network-metrics-daemon-srz9n","openshift-ovn-kubernetes/ovnkube-node-7x6bl","kube-system/konnectivity-agent-lfcks","kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq","openshift-multus/multus-additional-cni-plugins-jtdd2","openshift-network-diagnostics/network-check-target-nmhjf","openshift-network-operator/iptables-alerter-kxxsq"] Jul 07 18:45:00.340502 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.340475 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:00.340621 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.340602 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:00.342621 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.341556 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.343017 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.342998 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.344252 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.343932 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 18:45:00.344252 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.344161 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-r4gp8\"" Jul 07 18:45:00.344706 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.344690 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 18:45:00.345007 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.344989 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 18:45:00.345291 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.345273 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dvfp5\"" Jul 07 18:45:00.345961 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.345537 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 18:45:00.345961 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.345869 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 18:45:00.346152 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.346062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.346851 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.346834 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.347318 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.347301 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.348301 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.348278 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 18:45:00.349006 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.348528 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 18:45:00.349006 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.348695 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-grmxz\"" Jul 07 18:45:00.349006 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.348807 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:00.349006 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.348826 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 18:45:00.350081 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.349738 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 18:45:00.350081 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.349782 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:45:00.350081 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.349988 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 18:45:00.350330 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.350233 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rbjh8\"" Jul 07 18:45:00.350383 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.350366 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tg8zv\"" Jul 07 18:45:00.350447 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.350393 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 18:45:00.350575 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.350528 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 18:45:00.350700 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.350583 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:00.350700 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.350636 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:00.350908 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.350887 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 18:45:00.351670 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.351363 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 18:45:00.352047 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.351836 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 18:45:00.352047 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.351840 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 18:45:00.352184 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.352085 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.352876 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.352573 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 18:45:00.352876 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.352813 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7gxkb\"" Jul 07 18:45:00.353910 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.353893 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.354183 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.354163 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:45:00.354512 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.354410 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.354852 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.354459 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:00.355038 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.355016 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 18:45:00.355354 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.355333 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qswps\"" Jul 07 18:45:00.356804 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.356785 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 18:45:00.357364 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.357340 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 18:45:00.359062 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.357854 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q9vlw\"" Jul 07 18:45:00.359062 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.358186 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 18:45:00.359062 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.358346 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 18:45:00.360776 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.359557 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 18:45:00.360776 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.359615 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 18:45:00.362466 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.362448 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.363087 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363066 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-ovn\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.363179 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363099 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-tuned\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.363179 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363122 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-hosts-file\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.363179 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363146 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-slash\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.363179 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-etc-selinux\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.363376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363192 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f82p\" (UniqueName: \"kubernetes.io/projected/0d3043a3-d10f-4fef-bbe1-7f9d18666671-kube-api-access-9f82p\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.363376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363216 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-tmp\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.363376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-systemd-units\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.363376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363261 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-systemd\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.363376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363283 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-cni-bin\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.363376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363305 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8af537b1-f55e-42ec-9269-1f55dc81a5ec-kubelet-config\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.363376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363329 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-registration-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.363376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363364 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-cni-multus\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363386 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-kubernetes\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363408 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1090d31-86e7-4265-a03c-163e86d9e538-cni-binary-copy\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-k8s-cni-cncf-io\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363455 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363534 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysctl-d\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363559 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xljj\" (UniqueName: \"kubernetes.io/projected/6fc27e7d-10f3-4398-9097-15b0d53d19a0-kube-api-access-6xljj\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363581 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-run-netns\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363609 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovnkube-config\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363630 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d3043a3-d10f-4fef-bbe1-7f9d18666671-host\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363709 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d3043a3-d10f-4fef-bbe1-7f9d18666671-serviceca\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363733 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fc27e7d-10f3-4398-9097-15b0d53d19a0-host-slash\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.363764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-cni-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363778 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-log-socket\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfbkj\" (UniqueName: \"kubernetes.io/projected/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-kube-api-access-sfbkj\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363824 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-cnibin\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363871 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-kubelet\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363895 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysconfig\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363917 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-cni-netd\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363941 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-env-overrides\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363968 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-device-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.363997 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-run\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364019 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-os-release\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-multus-certs\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364074 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-socket-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364099 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysctl-conf\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fc27e7d-10f3-4398-9097-15b0d53d19a0-iptables-alerter-script\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.364353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364298 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-kubelet\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-etc-kubernetes\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364377 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-run-ovn-kubernetes\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364425 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364449 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-sys-fs\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364474 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-modprobe-d\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364497 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-sys\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364519 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-host\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364544 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbgb4\" (UniqueName: \"kubernetes.io/projected/cca766b7-da36-4ad6-95f3-3357c28ba6bf-kube-api-access-tbgb4\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364584 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-systemd\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364609 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-lib-modules\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364637 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97jf8\" (UniqueName: \"kubernetes.io/projected/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-kube-api-access-97jf8\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364684 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-tmp-dir\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364711 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-system-cni-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364742 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-hostroot\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364765 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1090d31-86e7-4265-a03c-163e86d9e538-multus-daemon-config\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365216 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364798 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ss5f6\"" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-node-log\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364832 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khc65\" (UniqueName: \"kubernetes.io/projected/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-kube-api-access-khc65\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-netns\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbf2q\" (UniqueName: \"kubernetes.io/projected/f1090d31-86e7-4265-a03c-163e86d9e538-kube-api-access-hbf2q\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364909 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8af537b1-f55e-42ec-9269-1f55dc81a5ec-dbus\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364933 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.364957 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d49lk\" (UniqueName: \"kubernetes.io/projected/8ebbdc88-6861-4d70-b974-80ed25f07a17-kube-api-access-d49lk\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-socket-dir-parent\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365034 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-conf-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365059 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b5c3ee7-abd0-4f1e-a55b-974c4614593a-agent-certs\") pod \"konnectivity-agent-lfcks\" (UID: \"6b5c3ee7-abd0-4f1e-a55b-974c4614593a\") " pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365089 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-etc-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365115 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovnkube-script-lib\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365139 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-cni-bin\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365167 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b5c3ee7-abd0-4f1e-a55b-974c4614593a-konnectivity-ca\") pod \"konnectivity-agent-lfcks\" (UID: \"6b5c3ee7-abd0-4f1e-a55b-974c4614593a\") " pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365194 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-var-lib-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365198 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 18:45:00.365960 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365216 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovn-node-metrics-cert\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.366597 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-var-lib-kubelet\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.366597 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.365360 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 18:45:00.367062 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.367042 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 18:45:00.390879 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.390856 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fmknn" Jul 07 18:45:00.396788 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.396766 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fmknn" Jul 07 18:45:00.428325 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.428280 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:45:00.453326 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.453301 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 18:45:00.465951 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.465924 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-systemd-units\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.465951 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.465951 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-systemd\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.466118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.465968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-cni-bin\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.466118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.465983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8af537b1-f55e-42ec-9269-1f55dc81a5ec-kubelet-config\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.466118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.465999 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-registration-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.466118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-cni-multus\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.466118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466044 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-kubernetes\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.466118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466050 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-systemd\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.466118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466068 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1090d31-86e7-4265-a03c-163e86d9e538-cni-binary-copy\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.466118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-k8s-cni-cncf-io\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466138 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466170 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466194 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysctl-d\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xljj\" (UniqueName: \"kubernetes.io/projected/6fc27e7d-10f3-4398-9097-15b0d53d19a0-kube-api-access-6xljj\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466243 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-run-netns\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovnkube-config\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466289 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d3043a3-d10f-4fef-bbe1-7f9d18666671-host\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466311 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d3043a3-d10f-4fef-bbe1-7f9d18666671-serviceca\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466334 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fc27e7d-10f3-4398-9097-15b0d53d19a0-host-slash\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466357 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-cni-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-log-socket\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfbkj\" (UniqueName: \"kubernetes.io/projected/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-kube-api-access-sfbkj\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-cnibin\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.466492 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466466 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h9dk\" (UniqueName: \"kubernetes.io/projected/107cfa3d-ccec-412f-987b-10b2b0c2174f-kube-api-access-5h9dk\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-kubelet\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysconfig\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466593 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-cni-netd\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466598 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1090d31-86e7-4265-a03c-163e86d9e538-cni-binary-copy\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466617 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-env-overrides\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-device-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466675 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-cni-bin\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466690 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-run\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466711 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8af537b1-f55e-42ec-9269-1f55dc81a5ec-kubelet-config\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466713 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-os-release\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466741 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-registration-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-multus-certs\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466780 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-kubernetes\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466787 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-multus-certs\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466806 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-log-socket\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466828 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-systemd-units\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.467192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466827 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-cni-multus\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466856 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-k8s-cni-cncf-io\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.466880 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.466949 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.466996 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret podName:8af537b1-f55e-42ec-9269-1f55dc81a5ec nodeName:}" failed. No retries permitted until 2026-07-07 18:45:00.966981032 +0000 UTC m=+2.992099087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret") pod "global-pull-secret-syncer-b4hlp" (UID: "8af537b1-f55e-42ec-9269-1f55dc81a5ec") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467136 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysctl-d\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467159 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-cnibin\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467283 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-run-netns\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467394 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-kubelet\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467451 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysconfig\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467525 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-cni-netd\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467669 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovnkube-config\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467720 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d3043a3-d10f-4fef-bbe1-7f9d18666671-host\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.467933 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-env-overrides\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.468035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468001 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-device-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468051 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-run\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468074 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d3043a3-d10f-4fef-bbe1-7f9d18666671-serviceca\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468112 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-os-release\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468129 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fc27e7d-10f3-4398-9097-15b0d53d19a0-host-slash\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468150 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-socket-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysctl-conf\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468186 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-cni-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fc27e7d-10f3-4398-9097-15b0d53d19a0-iptables-alerter-script\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468227 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-kubelet\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468250 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-etc-kubernetes\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468282 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-socket-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468308 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468337 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-run-ovn-kubernetes\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468390 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-sys-fs\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.468718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-modprobe-d\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468464 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-sys\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468487 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-host\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468511 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbgb4\" (UniqueName: \"kubernetes.io/projected/cca766b7-da36-4ad6-95f3-3357c28ba6bf-kube-api-access-tbgb4\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468533 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-systemd\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-lib-modules\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97jf8\" (UniqueName: \"kubernetes.io/projected/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-kube-api-access-97jf8\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468608 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-tmp-dir\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468632 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-system-cni-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468677 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-hostroot\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1090d31-86e7-4265-a03c-163e86d9e538-multus-daemon-config\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468725 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-node-log\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468730 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fc27e7d-10f3-4398-9097-15b0d53d19a0-iptables-alerter-script\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468755 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khc65\" (UniqueName: \"kubernetes.io/projected/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-kube-api-access-khc65\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468780 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-netns\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbf2q\" (UniqueName: \"kubernetes.io/projected/f1090d31-86e7-4265-a03c-163e86d9e538-kube-api-access-hbf2q\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468830 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.469554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468857 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8af537b1-f55e-42ec-9269-1f55dc81a5ec-dbus\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468860 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-sysctl-conf\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468881 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468906 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d49lk\" (UniqueName: \"kubernetes.io/projected/8ebbdc88-6861-4d70-b974-80ed25f07a17-kube-api-access-d49lk\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468913 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-systemd\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-socket-dir-parent\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468945 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-kubelet\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468959 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-conf-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468982 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b5c3ee7-abd0-4f1e-a55b-974c4614593a-agent-certs\") pod \"konnectivity-agent-lfcks\" (UID: \"6b5c3ee7-abd0-4f1e-a55b-974c4614593a\") " pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-os-release\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469021 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-run-ovn-kubernetes\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469035 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-etc-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovnkube-script-lib\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469083 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-cni-bin\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.469091 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b5c3ee7-abd0-4f1e-a55b-974c4614593a-konnectivity-ca\") pod \"konnectivity-agent-lfcks\" (UID: \"6b5c3ee7-abd0-4f1e-a55b-974c4614593a\") " pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.469131 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs podName:19e9d15b-2047-4bb5-846d-dd0ffe63d8cd nodeName:}" failed. No retries permitted until 2026-07-07 18:45:00.969117862 +0000 UTC m=+2.994235918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs") pod "network-metrics-daemon-srz9n" (UID: "19e9d15b-2047-4bb5-846d-dd0ffe63d8cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:00.470367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469150 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-var-lib-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469203 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovn-node-metrics-cert\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469230 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-var-lib-kubelet\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469254 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-cnibin\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-ovn\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469304 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-tuned\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469329 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-hosts-file\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469352 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-system-cni-dir\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-slash\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469407 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-etc-selinux\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469433 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f82p\" (UniqueName: \"kubernetes.io/projected/0d3043a3-d10f-4fef-bbe1-7f9d18666671-kube-api-access-9f82p\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469456 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-tmp\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469572 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-sys-fs\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469607 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b5c3ee7-abd0-4f1e-a55b-974c4614593a-konnectivity-ca\") pod \"konnectivity-agent-lfcks\" (UID: \"6b5c3ee7-abd0-4f1e-a55b-974c4614593a\") " pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469689 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-modprobe-d\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.471168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.468984 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-etc-kubernetes\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469728 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-var-lib-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469759 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-lib-modules\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469808 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-var-lib-cni-bin\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469824 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-socket-dir-parent\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469859 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-node-log\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469870 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-multus-conf-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.469986 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470130 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-tmp-dir\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470190 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovnkube-script-lib\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470238 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-ovn\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470308 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-system-cni-dir\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470434 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8af537b1-f55e-42ec-9269-1f55dc81a5ec-dbus\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470534 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-etc-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470604 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1090d31-86e7-4265-a03c-163e86d9e538-multus-daemon-config\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470715 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-var-lib-kubelet\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470783 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-sys\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470819 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-host-slash\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470843 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-host\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.471984 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470867 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-hosts-file\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.472871 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470217 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-hostroot\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.472871 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.470938 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1090d31-86e7-4265-a03c-163e86d9e538-host-run-netns\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.475848 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cca766b7-da36-4ad6-95f3-3357c28ba6bf-run-openvswitch\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.476359 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b5c3ee7-abd0-4f1e-a55b-974c4614593a-agent-certs\") pod \"konnectivity-agent-lfcks\" (UID: \"6b5c3ee7-abd0-4f1e-a55b-974c4614593a\") " pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.476724 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.476494 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-tmp\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.477482 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ebbdc88-6861-4d70-b974-80ed25f07a17-etc-selinux\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.477722 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-etc-tuned\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.478682 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.478702 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:45:00.478710 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.478717 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8l4b9 for pod openshift-network-diagnostics/network-check-target-nmhjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:00.479148 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.478774 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9 podName:2753fb64-3975-4312-96ea-9cf482c0081c nodeName:}" failed. No retries permitted until 2026-07-07 18:45:00.978762189 +0000 UTC m=+3.003880253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8l4b9" (UniqueName: "kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9") pod "network-check-target-nmhjf" (UID: "2753fb64-3975-4312-96ea-9cf482c0081c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:00.481567 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.481529 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbgb4\" (UniqueName: \"kubernetes.io/projected/cca766b7-da36-4ad6-95f3-3357c28ba6bf-kube-api-access-tbgb4\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.483199 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.482524 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cca766b7-da36-4ad6-95f3-3357c28ba6bf-ovn-node-metrics-cert\") pod \"ovnkube-node-7x6bl\" (UID: \"cca766b7-da36-4ad6-95f3-3357c28ba6bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.483845 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.483821 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f82p\" (UniqueName: \"kubernetes.io/projected/0d3043a3-d10f-4fef-bbe1-7f9d18666671-kube-api-access-9f82p\") pod \"node-ca-f8lmc\" (UID: \"0d3043a3-d10f-4fef-bbe1-7f9d18666671\") " pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.484227 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.484206 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfbkj\" (UniqueName: \"kubernetes.io/projected/039885f1-e2ef-4c36-8fe1-c1b0a1a023ea-kube-api-access-sfbkj\") pod \"node-resolver-5wwvt\" (UID: \"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea\") " pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.484625 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.484606 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97jf8\" (UniqueName: \"kubernetes.io/projected/6cb3691f-eca0-4dbd-9cd9-133f8ffcd439-kube-api-access-97jf8\") pod \"tuned-w2bwn\" (UID: \"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439\") " pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.484938 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.484895 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xljj\" (UniqueName: \"kubernetes.io/projected/6fc27e7d-10f3-4398-9097-15b0d53d19a0-kube-api-access-6xljj\") pod \"iptables-alerter-kxxsq\" (UID: \"6fc27e7d-10f3-4398-9097-15b0d53d19a0\") " pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.485544 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.485504 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khc65\" (UniqueName: \"kubernetes.io/projected/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-kube-api-access-khc65\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:00.487609 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.487591 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d49lk\" (UniqueName: \"kubernetes.io/projected/8ebbdc88-6861-4d70-b974-80ed25f07a17-kube-api-access-d49lk\") pod \"aws-ebs-csi-driver-node-npcdq\" (UID: \"8ebbdc88-6861-4d70-b974-80ed25f07a17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.489180 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.489158 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbf2q\" (UniqueName: \"kubernetes.io/projected/f1090d31-86e7-4265-a03c-163e86d9e538-kube-api-access-hbf2q\") pod \"multus-dsl26\" (UID: \"f1090d31-86e7-4265-a03c-163e86d9e538\") " pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.570320 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-os-release\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570320 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570510 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570353 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-cnibin\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570510 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570378 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-system-cni-dir\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570510 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570404 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570510 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h9dk\" (UniqueName: \"kubernetes.io/projected/107cfa3d-ccec-412f-987b-10b2b0c2174f-kube-api-access-5h9dk\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570510 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570498 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570751 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570535 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570751 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570721 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-system-cni-dir\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570857 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570765 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-cnibin\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.570857 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.570833 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-os-release\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.571150 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.571121 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/107cfa3d-ccec-412f-987b-10b2b0c2174f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.571150 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.571143 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.571326 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.571180 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.571434 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.571411 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/107cfa3d-ccec-412f-987b-10b2b0c2174f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.583639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.583569 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h9dk\" (UniqueName: \"kubernetes.io/projected/107cfa3d-ccec-412f-987b-10b2b0c2174f-kube-api-access-5h9dk\") pod \"multus-additional-cni-plugins-jtdd2\" (UID: \"107cfa3d-ccec-412f-987b-10b2b0c2174f\") " pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.585367 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.585318 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" event={"ID":"1d15639f345b044a83529457ff7d48d9","Type":"ContainerStarted","Data":"a1ca130896024ec08ccd8812219a22a1cc9653b1e5aaaa5424e097211287288a"} Jul 07 18:45:00.586982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.586934 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" event={"ID":"5b50e609bf668c243e15e77d050b97b9","Type":"ContainerStarted","Data":"b8de1c9c156684fa1cf5a33a2e1c856fa4709fdf3d14c4a0b57413acee4f6367"} Jul 07 18:45:00.657564 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.657527 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" Jul 07 18:45:00.669063 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.669034 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5wwvt" Jul 07 18:45:00.676912 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:45:00.676887 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039885f1_e2ef_4c36_8fe1_c1b0a1a023ea.slice/crio-b895df9e6a76a7720231427948dc1fa8ec936461d99afaad66080f923bec1ddd WatchSource:0}: Error finding container b895df9e6a76a7720231427948dc1fa8ec936461d99afaad66080f923bec1ddd: Status 404 returned error can't find the container with id b895df9e6a76a7720231427948dc1fa8ec936461d99afaad66080f923bec1ddd Jul 07 18:45:00.678937 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.678786 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8lmc" Jul 07 18:45:00.686694 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.686672 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kxxsq" Jul 07 18:45:00.694633 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.694614 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dsl26" Jul 07 18:45:00.702412 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.702391 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:00.711156 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.711138 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" Jul 07 18:45:00.718056 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.718034 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:00.724842 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.724823 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" Jul 07 18:45:00.972995 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.972912 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:00.972995 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:00.972988 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:00.973205 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.973096 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:00.973205 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.973098 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:00.973205 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.973159 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret podName:8af537b1-f55e-42ec-9269-1f55dc81a5ec nodeName:}" failed. No retries permitted until 2026-07-07 18:45:01.973141779 +0000 UTC m=+3.998259836 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret") pod "global-pull-secret-syncer-b4hlp" (UID: "8af537b1-f55e-42ec-9269-1f55dc81a5ec") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:00.973205 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:00.973175 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs podName:19e9d15b-2047-4bb5-846d-dd0ffe63d8cd nodeName:}" failed. No retries permitted until 2026-07-07 18:45:01.973167889 +0000 UTC m=+3.998285940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs") pod "network-metrics-daemon-srz9n" (UID: "19e9d15b-2047-4bb5-846d-dd0ffe63d8cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:01.074107 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:01.074068 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:01.074296 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.074273 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:45:01.074359 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.074306 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:45:01.074359 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.074322 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8l4b9 for pod openshift-network-diagnostics/network-check-target-nmhjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:01.074438 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.074380 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9 podName:2753fb64-3975-4312-96ea-9cf482c0081c nodeName:}" failed. No retries permitted until 2026-07-07 18:45:02.074365711 +0000 UTC m=+4.099483762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8l4b9" (UniqueName: "kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9") pod "network-check-target-nmhjf" (UID: "2753fb64-3975-4312-96ea-9cf482c0081c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:01.399047 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:01.398999 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 18:40:00 +0000 UTC" deadline="2027-12-06 20:09:13.889340427 +0000 UTC" Jul 07 18:45:01.399047 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:01.399042 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12409h24m12.49030398s" Jul 07 18:45:01.579087 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:01.579052 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:01.579246 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.579177 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:01.589596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:01.589560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5wwvt" event={"ID":"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea","Type":"ContainerStarted","Data":"b895df9e6a76a7720231427948dc1fa8ec936461d99afaad66080f923bec1ddd"} Jul 07 18:45:01.590762 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:01.590736 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" event={"ID":"8ebbdc88-6861-4d70-b974-80ed25f07a17","Type":"ContainerStarted","Data":"62c40e8a24bd740d3abd12778498036d371f398fa902a782651697898331628a"} Jul 07 18:45:01.690038 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:45:01.689998 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc27e7d_10f3_4398_9097_15b0d53d19a0.slice/crio-0e56924fc5cbf1bd29f88fcabee42a73a128e04602bc03613f253b73674bdb19 WatchSource:0}: Error finding container 0e56924fc5cbf1bd29f88fcabee42a73a128e04602bc03613f253b73674bdb19: Status 404 returned error can't find the container with id 0e56924fc5cbf1bd29f88fcabee42a73a128e04602bc03613f253b73674bdb19 Jul 07 18:45:01.690773 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:45:01.690753 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb3691f_eca0_4dbd_9cd9_133f8ffcd439.slice/crio-049dcaf8ad1a9a7ac6550f9831981164e572f95f9212f21dadfd653f2a198f9c WatchSource:0}: Error finding container 049dcaf8ad1a9a7ac6550f9831981164e572f95f9212f21dadfd653f2a198f9c: Status 404 returned error can't find the container with id 049dcaf8ad1a9a7ac6550f9831981164e572f95f9212f21dadfd653f2a198f9c Jul 07 18:45:01.692970 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:45:01.692815 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3043a3_d10f_4fef_bbe1_7f9d18666671.slice/crio-d24fc83b8dd0d7a3e552a4d77b24dc07ff4ad1f23e3d00678d2bb225146e23c7 WatchSource:0}: Error finding container d24fc83b8dd0d7a3e552a4d77b24dc07ff4ad1f23e3d00678d2bb225146e23c7: Status 404 returned error can't find the container with id d24fc83b8dd0d7a3e552a4d77b24dc07ff4ad1f23e3d00678d2bb225146e23c7 Jul 07 18:45:01.702669 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:45:01.702495 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1090d31_86e7_4265_a03c_163e86d9e538.slice/crio-ec9e0233112c58f3cabdbc98994f365282f2843859050bd5d7bc40a88c256e26 WatchSource:0}: Error finding container ec9e0233112c58f3cabdbc98994f365282f2843859050bd5d7bc40a88c256e26: Status 404 returned error can't find the container with id ec9e0233112c58f3cabdbc98994f365282f2843859050bd5d7bc40a88c256e26 Jul 07 18:45:01.705070 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:45:01.705047 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5c3ee7_abd0_4f1e_a55b_974c4614593a.slice/crio-ff0f4863e323e40e7b85a41479cf229691843c5546d6f926d31009723269b347 WatchSource:0}: Error finding container ff0f4863e323e40e7b85a41479cf229691843c5546d6f926d31009723269b347: Status 404 returned error can't find the container with id ff0f4863e323e40e7b85a41479cf229691843c5546d6f926d31009723269b347 Jul 07 18:45:01.981236 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:01.980981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:01.981376 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:01.981290 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:01.981432 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.981144 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:01.981495 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.981459 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs podName:19e9d15b-2047-4bb5-846d-dd0ffe63d8cd nodeName:}" failed. No retries permitted until 2026-07-07 18:45:03.981441969 +0000 UTC m=+6.006560021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs") pod "network-metrics-daemon-srz9n" (UID: "19e9d15b-2047-4bb5-846d-dd0ffe63d8cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:01.981495 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.981398 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:01.981599 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:01.981503 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret podName:8af537b1-f55e-42ec-9269-1f55dc81a5ec nodeName:}" failed. No retries permitted until 2026-07-07 18:45:03.981492682 +0000 UTC m=+6.006610747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret") pod "global-pull-secret-syncer-b4hlp" (UID: "8af537b1-f55e-42ec-9269-1f55dc81a5ec") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:02.081893 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.081850 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:02.082071 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:02.082048 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:45:02.082119 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:02.082077 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:45:02.082119 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:02.082091 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8l4b9 for pod openshift-network-diagnostics/network-check-target-nmhjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:02.082195 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:02.082154 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9 podName:2753fb64-3975-4312-96ea-9cf482c0081c nodeName:}" failed. No retries permitted until 2026-07-07 18:45:04.082135187 +0000 UTC m=+6.107253243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8l4b9" (UniqueName: "kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9") pod "network-check-target-nmhjf" (UID: "2753fb64-3975-4312-96ea-9cf482c0081c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:02.399338 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.399195 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 18:40:00 +0000 UTC" deadline="2027-12-27 17:01:59.353695859 +0000 UTC" Jul 07 18:45:02.399338 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.399292 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12910h16m56.954411345s" Jul 07 18:45:02.581548 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.581516 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:02.581781 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:02.581657 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:02.582129 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.582106 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:02.582216 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:02.582197 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:02.595908 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.595875 2582 generic.go:356] "Generic (PLEG): container finished" podID="1d15639f345b044a83529457ff7d48d9" containerID="614d3f1fe6f6b8c69f1087a96998765cb64d19c82b81f201ee8c1fd054e7d1cf" exitCode=0 Jul 07 18:45:02.596172 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.596145 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" event={"ID":"1d15639f345b044a83529457ff7d48d9","Type":"ContainerDied","Data":"614d3f1fe6f6b8c69f1087a96998765cb64d19c82b81f201ee8c1fd054e7d1cf"} Jul 07 18:45:02.606196 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.606116 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" event={"ID":"107cfa3d-ccec-412f-987b-10b2b0c2174f","Type":"ContainerStarted","Data":"1d36739587651a09e7013ecdf1b242d82ed0156e9de083a0228bfe36cce8da46"} Jul 07 18:45:02.614026 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.613817 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"9a15faae3b917d94b39f6b75d95a52032b7845f9f3e62773b20d13c10e47d5b4"} Jul 07 18:45:02.616234 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.616185 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8lmc" event={"ID":"0d3043a3-d10f-4fef-bbe1-7f9d18666671","Type":"ContainerStarted","Data":"d24fc83b8dd0d7a3e552a4d77b24dc07ff4ad1f23e3d00678d2bb225146e23c7"} Jul 07 18:45:02.617877 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.617854 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kxxsq" event={"ID":"6fc27e7d-10f3-4398-9097-15b0d53d19a0","Type":"ContainerStarted","Data":"0e56924fc5cbf1bd29f88fcabee42a73a128e04602bc03613f253b73674bdb19"} Jul 07 18:45:02.621438 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.621413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" event={"ID":"5b50e609bf668c243e15e77d050b97b9","Type":"ContainerStarted","Data":"d417e93b6889ade9f154261efe1ff68bd519cc70dbc2bdf5f9d401abbd986f6d"} Jul 07 18:45:02.623588 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.623565 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lfcks" event={"ID":"6b5c3ee7-abd0-4f1e-a55b-974c4614593a","Type":"ContainerStarted","Data":"ff0f4863e323e40e7b85a41479cf229691843c5546d6f926d31009723269b347"} Jul 07 18:45:02.624948 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.624923 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dsl26" event={"ID":"f1090d31-86e7-4265-a03c-163e86d9e538","Type":"ContainerStarted","Data":"ec9e0233112c58f3cabdbc98994f365282f2843859050bd5d7bc40a88c256e26"} Jul 07 18:45:02.626977 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:02.626939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" event={"ID":"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439","Type":"ContainerStarted","Data":"049dcaf8ad1a9a7ac6550f9831981164e572f95f9212f21dadfd653f2a198f9c"} Jul 07 18:45:03.579597 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:03.579556 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:03.580108 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:03.579727 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:03.647759 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:03.647722 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" event={"ID":"1d15639f345b044a83529457ff7d48d9","Type":"ContainerStarted","Data":"200491de8b72b526c7fb92110a636facbbab5e21c2b6ca257edc26b9981270d3"} Jul 07 18:45:03.668782 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:03.668721 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-163.ec2.internal" podStartSLOduration=4.668700881 podStartE2EDuration="4.668700881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:02.63872561 +0000 UTC m=+4.663843697" watchObservedRunningTime="2026-07-07 18:45:03.668700881 +0000 UTC m=+5.693818953" Jul 07 18:45:03.669361 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:03.669314 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-163.ec2.internal" podStartSLOduration=4.669305602 podStartE2EDuration="4.669305602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:03.6683583 +0000 UTC m=+5.693476373" watchObservedRunningTime="2026-07-07 18:45:03.669305602 +0000 UTC m=+5.694423677" Jul 07 18:45:03.996768 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:03.996677 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:03.996920 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:03.996768 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:03.996920 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:03.996842 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:03.996920 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:03.996886 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:03.996920 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:03.996911 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs podName:19e9d15b-2047-4bb5-846d-dd0ffe63d8cd nodeName:}" failed. No retries permitted until 2026-07-07 18:45:07.996893383 +0000 UTC m=+10.022011441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs") pod "network-metrics-daemon-srz9n" (UID: "19e9d15b-2047-4bb5-846d-dd0ffe63d8cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:03.997136 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:03.996938 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret podName:8af537b1-f55e-42ec-9269-1f55dc81a5ec nodeName:}" failed. No retries permitted until 2026-07-07 18:45:07.996920179 +0000 UTC m=+10.022038232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret") pod "global-pull-secret-syncer-b4hlp" (UID: "8af537b1-f55e-42ec-9269-1f55dc81a5ec") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:04.098570 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:04.097983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:04.098570 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:04.098143 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:45:04.098570 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:04.098161 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:45:04.098570 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:04.098173 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8l4b9 for pod openshift-network-diagnostics/network-check-target-nmhjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:04.098570 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:04.098230 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9 podName:2753fb64-3975-4312-96ea-9cf482c0081c nodeName:}" failed. No retries permitted until 2026-07-07 18:45:08.09821247 +0000 UTC m=+10.123330524 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8l4b9" (UniqueName: "kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9") pod "network-check-target-nmhjf" (UID: "2753fb64-3975-4312-96ea-9cf482c0081c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:04.579425 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:04.578811 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:04.579425 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:04.578937 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:04.579425 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:04.579309 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:04.579425 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:04.579390 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:05.578664 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:05.578622 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:05.578837 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:05.578749 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:06.579870 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:06.579749 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:06.580253 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:06.579876 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:06.580253 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:06.580218 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:06.580348 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:06.580314 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:07.579631 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:07.579594 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:07.579822 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:07.579744 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:08.029974 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:08.029893 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:08.030405 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:08.029977 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:08.030405 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.030066 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:08.030405 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.030091 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:08.030405 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.030127 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret podName:8af537b1-f55e-42ec-9269-1f55dc81a5ec nodeName:}" failed. No retries permitted until 2026-07-07 18:45:16.030112752 +0000 UTC m=+18.055230808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret") pod "global-pull-secret-syncer-b4hlp" (UID: "8af537b1-f55e-42ec-9269-1f55dc81a5ec") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:08.030405 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.030150 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs podName:19e9d15b-2047-4bb5-846d-dd0ffe63d8cd nodeName:}" failed. No retries permitted until 2026-07-07 18:45:16.030132801 +0000 UTC m=+18.055250852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs") pod "network-metrics-daemon-srz9n" (UID: "19e9d15b-2047-4bb5-846d-dd0ffe63d8cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:08.130964 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:08.130927 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:08.131143 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.131116 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:45:08.131143 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.131142 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:45:08.131343 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.131155 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8l4b9 for pod openshift-network-diagnostics/network-check-target-nmhjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:08.131343 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.131211 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9 podName:2753fb64-3975-4312-96ea-9cf482c0081c nodeName:}" failed. No retries permitted until 2026-07-07 18:45:16.131192727 +0000 UTC m=+18.156310783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8l4b9" (UniqueName: "kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9") pod "network-check-target-nmhjf" (UID: "2753fb64-3975-4312-96ea-9cf482c0081c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:08.579580 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:08.579542 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:08.579750 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.579672 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:08.579750 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:08.579721 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:08.579873 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:08.579837 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:09.578909 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:09.578814 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:09.579348 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:09.578966 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:10.583710 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:10.583683 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:10.584163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:10.583691 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:10.584163 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:10.583795 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:10.584163 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:10.583877 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:11.579013 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:11.578975 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:11.579206 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:11.579094 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:12.580717 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.580483 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:12.581154 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.580553 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:12.581154 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:12.580808 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:12.581154 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:12.580897 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:12.667178 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.667094 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" event={"ID":"6cb3691f-eca0-4dbd-9cd9-133f8ffcd439","Type":"ContainerStarted","Data":"0680202cd1986d4643cd4e2ad6ec745d3ac5a08c3730fac857110a43cc004622"} Jul 07 18:45:12.668803 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.668773 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5wwvt" event={"ID":"039885f1-e2ef-4c36-8fe1-c1b0a1a023ea","Type":"ContainerStarted","Data":"60c0fffb2e5d608ddb31f4520b872a15557bddb02ed745cd3cd651dadf8f6316"} Jul 07 18:45:12.670375 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.670346 2582 generic.go:356] "Generic (PLEG): container finished" podID="107cfa3d-ccec-412f-987b-10b2b0c2174f" containerID="8a955b85f126eed50394843e9e21b0874ad3cb59e1403a17806a7af8bc1c5c21" exitCode=0 Jul 07 18:45:12.670487 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.670428 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" event={"ID":"107cfa3d-ccec-412f-987b-10b2b0c2174f","Type":"ContainerDied","Data":"8a955b85f126eed50394843e9e21b0874ad3cb59e1403a17806a7af8bc1c5c21"} Jul 07 18:45:12.672082 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.672057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8lmc" event={"ID":"0d3043a3-d10f-4fef-bbe1-7f9d18666671","Type":"ContainerStarted","Data":"08c8609252939dd1225cc4c548abef8d36591e87da28dcdf92301c364912bd76"} Jul 07 18:45:12.673622 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.673563 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" event={"ID":"8ebbdc88-6861-4d70-b974-80ed25f07a17","Type":"ContainerStarted","Data":"e6663f2e0841fa4d37f3d49eede425d6cdcc4d101f1c898bc4a9f3e1e2368dbd"} Jul 07 18:45:12.675019 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.675001 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lfcks" event={"ID":"6b5c3ee7-abd0-4f1e-a55b-974c4614593a","Type":"ContainerStarted","Data":"7b0634a77df420e21f02651fdab84245fdb0614b9faf20b5938ab6910e0de45c"} Jul 07 18:45:12.707103 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.707047 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w2bwn" podStartSLOduration=4.368474141 podStartE2EDuration="14.707030846s" totalImagesPullingTime="10.338556705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:12.684485066 +0000 UTC m=+14.709603143" watchObservedRunningTime="2026-07-07 18:45:12.707030846 +0000 UTC m=+14.732148920" Jul 07 18:45:12.721776 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.721728 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5wwvt" podStartSLOduration=3.713963985 podStartE2EDuration="14.721709926s" totalImagesPullingTime="11.007745941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:12.721070649 +0000 UTC m=+14.746188722" watchObservedRunningTime="2026-07-07 18:45:12.721709926 +0000 UTC m=+14.746828001" Jul 07 18:45:12.736319 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:12.736265 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f8lmc" podStartSLOduration=4.671022052 podStartE2EDuration="14.736246379s" totalImagesPullingTime="10.065224327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:12.735695762 +0000 UTC m=+14.760813836" watchObservedRunningTime="2026-07-07 18:45:12.736246379 +0000 UTC m=+14.761364450" Jul 07 18:45:13.579423 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:13.579389 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:13.579595 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:13.579525 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:13.679951 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:13.679911 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kxxsq" event={"ID":"6fc27e7d-10f3-4398-9097-15b0d53d19a0","Type":"ContainerStarted","Data":"a9be0100d12aa781ef7bb5213b93b06948c3254a6dc18ac47d728f2423cbf377"} Jul 07 18:45:13.699426 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:13.699370 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lfcks" podStartSLOduration=5.560713399 podStartE2EDuration="15.699349587s" totalImagesPullingTime="10.138636188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:12.752099653 +0000 UTC m=+14.777217727" watchObservedRunningTime="2026-07-07 18:45:13.699349587 +0000 UTC m=+15.724467662" Jul 07 18:45:13.699845 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:13.699774 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kxxsq" podStartSLOduration=5.665763025 podStartE2EDuration="15.699762729s" totalImagesPullingTime="10.033999704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:13.698796616 +0000 UTC m=+15.723914690" watchObservedRunningTime="2026-07-07 18:45:13.699762729 +0000 UTC m=+15.724880805" Jul 07 18:45:14.352306 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:14.352223 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:14.578791 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:14.578753 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:14.578952 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:14.578763 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:14.578952 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:14.578873 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:14.579085 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:14.578964 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:15.578893 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:15.578860 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:15.579464 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:15.578999 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:15.732164 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:15.732134 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:15.735469 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:15.735436 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:16.090638 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:16.090599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:16.090912 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:16.090689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:16.090912 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.090754 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:16.090912 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.090772 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:16.090912 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.090826 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret podName:8af537b1-f55e-42ec-9269-1f55dc81a5ec nodeName:}" failed. No retries permitted until 2026-07-07 18:45:32.090807696 +0000 UTC m=+34.115925748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret") pod "global-pull-secret-syncer-b4hlp" (UID: "8af537b1-f55e-42ec-9269-1f55dc81a5ec") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:16.090912 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.090843 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs podName:19e9d15b-2047-4bb5-846d-dd0ffe63d8cd nodeName:}" failed. No retries permitted until 2026-07-07 18:45:32.090834368 +0000 UTC m=+34.115952422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs") pod "network-metrics-daemon-srz9n" (UID: "19e9d15b-2047-4bb5-846d-dd0ffe63d8cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:16.191641 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:16.191593 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:16.191825 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.191774 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:45:16.191825 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.191800 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:45:16.191825 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.191813 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8l4b9 for pod openshift-network-diagnostics/network-check-target-nmhjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:16.191950 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.191870 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9 podName:2753fb64-3975-4312-96ea-9cf482c0081c nodeName:}" failed. No retries permitted until 2026-07-07 18:45:32.191851221 +0000 UTC m=+34.216969276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8l4b9" (UniqueName: "kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9") pod "network-check-target-nmhjf" (UID: "2753fb64-3975-4312-96ea-9cf482c0081c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:16.579067 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:16.579036 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:16.579716 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.579197 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:16.579716 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:16.579256 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:16.579716 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:16.579378 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:16.686226 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:16.686198 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lfcks" Jul 07 18:45:17.579390 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:17.579361 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:17.579893 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:17.579474 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:18.579979 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:18.579938 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:18.580436 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:18.580043 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:18.580436 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:18.580125 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:18.580436 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:18.580238 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:19.579344 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:19.579307 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:19.579529 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:19.579440 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:20.579190 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:20.579158 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:20.579713 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:20.579163 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:20.579713 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:20.579302 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:20.579713 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:20.579360 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:21.579370 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:21.579336 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:21.579743 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:21.579442 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:22.579272 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:22.579239 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:22.579446 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:22.579358 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:22.579773 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:22.579439 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:22.579773 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:22.579549 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:23.505359 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.505189 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 18:45:23.578834 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.578809 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:23.578939 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:23.578919 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:23.700034 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.700014 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:45:23.700416 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.700373 2582 generic.go:356] "Generic (PLEG): container finished" podID="cca766b7-da36-4ad6-95f3-3357c28ba6bf" containerID="99bb44d4ad5cc4a68ebffa3d468d71c94814022d561be4a0e5e8dc808c857fa3" exitCode=1 Jul 07 18:45:23.700481 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.700438 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerDied","Data":"99bb44d4ad5cc4a68ebffa3d468d71c94814022d561be4a0e5e8dc808c857fa3"} Jul 07 18:45:23.700537 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.700505 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"c6c28a089f055237425d4f9720171a9b7006db6f2a9ce70e8457d1020b171008"} Jul 07 18:45:23.702954 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.702907 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" event={"ID":"8ebbdc88-6861-4d70-b974-80ed25f07a17","Type":"ContainerStarted","Data":"a56c9e3c56a7d2c29c76a6540bfe04a709a6f6f967c4b940276bc5a3976116a3"} Jul 07 18:45:23.705732 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.705709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dsl26" event={"ID":"f1090d31-86e7-4265-a03c-163e86d9e538","Type":"ContainerStarted","Data":"c7a4141e30d50dec547a15a23c1184c611ca2433f61bc8ef50bbd703c3e0f0ce"} Jul 07 18:45:23.725487 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:23.725446 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dsl26" podStartSLOduration=5.646489033 podStartE2EDuration="25.725434393s" totalImagesPullingTime="20.07894536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:23.724954773 +0000 UTC m=+25.750072848" watchObservedRunningTime="2026-07-07 18:45:23.725434393 +0000 UTC m=+25.750552467" Jul 07 18:45:24.430307 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.430078 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T18:45:23.505350888Z","UUID":"af3b1ee9-0c24-41a7-905e-e7bcbb3be6dc","Handler":null,"Name":"","Endpoint":""} Jul 07 18:45:24.431598 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.431580 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 18:45:24.431728 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.431612 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 18:45:24.579460 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.579429 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:24.579460 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.579443 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:24.579710 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:24.579526 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:24.579710 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:24.579682 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:24.709513 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.709443 2582 generic.go:356] "Generic (PLEG): container finished" podID="107cfa3d-ccec-412f-987b-10b2b0c2174f" containerID="f46d2cbd78049de7ed1fe57268b2c81d5f012f721e024a26a30e323e20052443" exitCode=0 Jul 07 18:45:24.710012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.709518 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" event={"ID":"107cfa3d-ccec-412f-987b-10b2b0c2174f","Type":"ContainerDied","Data":"f46d2cbd78049de7ed1fe57268b2c81d5f012f721e024a26a30e323e20052443"} Jul 07 18:45:24.712153 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.712134 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:45:24.712526 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.712504 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"3375e590d815da12e89c4acde34b334cc090d795fc6683848fee52e88b83a52d"} Jul 07 18:45:24.712585 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.712563 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"a9d9b23b126470098b23eb0fc131f9534374b12b86a2ebc68337addca8eb41a6"} Jul 07 18:45:24.712619 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.712592 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"ce9ca67404a98d9c26d7f8bdb69f218bc9ca47ab7e81a7a9a49fce49f7dc9846"} Jul 07 18:45:24.712673 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:24.712618 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"7dd95d27e28083c2f37c5e04b25ff638ec077a377d31e212218554dc592b6041"} Jul 07 18:45:25.578943 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:25.578903 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:25.579105 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:25.579022 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:25.716953 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:25.716908 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" event={"ID":"8ebbdc88-6861-4d70-b974-80ed25f07a17","Type":"ContainerStarted","Data":"09a33013b91e05c63f86a28be6cf496696b9e824353d06d321c4c1c831e4bdd5"} Jul 07 18:45:25.735576 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:25.735525 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-npcdq" podStartSLOduration=5.618365657 podStartE2EDuration="27.735507684s" totalImagesPullingTime="22.117142027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:25.73518114 +0000 UTC m=+27.760299245" watchObservedRunningTime="2026-07-07 18:45:25.735507684 +0000 UTC m=+27.760625757" Jul 07 18:45:26.578911 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:26.578871 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:26.579070 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:26.578880 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:26.579070 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:26.579004 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:26.579070 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:26.579038 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:26.720558 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:26.720483 2582 generic.go:356] "Generic (PLEG): container finished" podID="107cfa3d-ccec-412f-987b-10b2b0c2174f" containerID="eda22d7f269b711edd128bbf4fba1632093139613514ea5c60db0ec32e7271d9" exitCode=0 Jul 07 18:45:26.720558 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:26.720548 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" event={"ID":"107cfa3d-ccec-412f-987b-10b2b0c2174f","Type":"ContainerDied","Data":"eda22d7f269b711edd128bbf4fba1632093139613514ea5c60db0ec32e7271d9"} Jul 07 18:45:26.723635 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:26.723621 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:45:26.724088 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:26.724069 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"486321639b84d132b87896f68d621856ed96178721d0df0d40b4b287a604e236"} Jul 07 18:45:27.578761 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:27.578732 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:27.578993 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:27.578840 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:28.579759 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.579575 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:28.580152 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.579666 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:28.580152 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:28.579827 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:28.580152 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:28.579928 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:28.730930 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.730828 2582 generic.go:356] "Generic (PLEG): container finished" podID="107cfa3d-ccec-412f-987b-10b2b0c2174f" containerID="87ab7309a5a1861e612aa30fe3577dc91c39910d1447e8f91a825e5801effa58" exitCode=0 Jul 07 18:45:28.730930 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.730898 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" event={"ID":"107cfa3d-ccec-412f-987b-10b2b0c2174f","Type":"ContainerDied","Data":"87ab7309a5a1861e612aa30fe3577dc91c39910d1447e8f91a825e5801effa58"} Jul 07 18:45:28.734522 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.734499 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:45:28.734895 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.734865 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"71bc315de5a972b47667aa1fd9ce4ca4caccf01df01579b06660989f69e0e3b4"} Jul 07 18:45:28.735227 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.735200 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:28.735467 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.735308 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:28.735467 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.735367 2582 scope.go:117] "RemoveContainer" containerID="99bb44d4ad5cc4a68ebffa3d468d71c94814022d561be4a0e5e8dc808c857fa3" Jul 07 18:45:28.754597 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:28.754574 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:29.579537 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:29.579506 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:29.579724 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:29.579627 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:29.744867 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:29.744823 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:45:29.745323 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:29.745183 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" event={"ID":"cca766b7-da36-4ad6-95f3-3357c28ba6bf","Type":"ContainerStarted","Data":"b9a7a48d04a2e19fa69e42834294c1e43397a98be860faf9393f7b609a32f64f"} Jul 07 18:45:29.745510 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:29.745426 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:29.774567 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:29.774530 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:45:29.780066 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:29.780016 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" podStartSLOduration=10.083098175 podStartE2EDuration="31.779996915s" totalImagesPullingTime="21.69689874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:29.779512197 +0000 UTC m=+31.804630272" watchObservedRunningTime="2026-07-07 18:45:29.779996915 +0000 UTC m=+31.805114989" Jul 07 18:45:30.578920 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:30.578889 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:30.578920 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:30.578910 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:30.579115 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:30.579009 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:30.579175 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:30.579148 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:30.744506 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:30.744472 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nmhjf"] Jul 07 18:45:30.746988 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:30.746959 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-srz9n"] Jul 07 18:45:30.747407 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:30.747085 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:30.747407 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:30.747193 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:30.747758 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:30.747740 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:30.747834 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:30.747819 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:30.757921 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:30.757887 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b4hlp"] Jul 07 18:45:30.758028 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:30.757973 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:30.758088 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:30.758067 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:32.111251 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:32.111216 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:32.111731 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:32.111266 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:32.111731 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.111377 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:32.111731 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.111380 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:32.111731 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.111432 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs podName:19e9d15b-2047-4bb5-846d-dd0ffe63d8cd nodeName:}" failed. No retries permitted until 2026-07-07 18:46:04.111415937 +0000 UTC m=+66.136534008 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs") pod "network-metrics-daemon-srz9n" (UID: "19e9d15b-2047-4bb5-846d-dd0ffe63d8cd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:45:32.111731 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.111449 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret podName:8af537b1-f55e-42ec-9269-1f55dc81a5ec nodeName:}" failed. No retries permitted until 2026-07-07 18:46:04.111440523 +0000 UTC m=+66.136558581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret") pod "global-pull-secret-syncer-b4hlp" (UID: "8af537b1-f55e-42ec-9269-1f55dc81a5ec") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:45:32.212257 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:32.212223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:32.212443 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.212415 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:45:32.212443 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.212446 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:45:32.212607 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.212459 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8l4b9 for pod openshift-network-diagnostics/network-check-target-nmhjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:32.212607 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.212524 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9 podName:2753fb64-3975-4312-96ea-9cf482c0081c nodeName:}" failed. No retries permitted until 2026-07-07 18:46:04.212505694 +0000 UTC m=+66.237623762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8l4b9" (UniqueName: "kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9") pod "network-check-target-nmhjf" (UID: "2753fb64-3975-4312-96ea-9cf482c0081c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:45:32.578749 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:32.578723 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:32.579054 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:32.578752 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:32.579054 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:32.578723 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:32.579054 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.578848 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:32.579054 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.578998 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:32.579297 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:32.579101 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:34.582146 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:34.581956 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:34.582574 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:34.581994 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:34.582574 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:34.582225 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:34.582574 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:34.582311 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:34.582574 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:34.582004 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:34.582574 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:34.582403 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:34.759256 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:34.759224 2582 generic.go:356] "Generic (PLEG): container finished" podID="107cfa3d-ccec-412f-987b-10b2b0c2174f" containerID="124c35b240c317161c625aaa46373974c8f887b8210ab2c8674a9671caae965f" exitCode=0 Jul 07 18:45:34.759410 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:34.759267 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" event={"ID":"107cfa3d-ccec-412f-987b-10b2b0c2174f","Type":"ContainerDied","Data":"124c35b240c317161c625aaa46373974c8f887b8210ab2c8674a9671caae965f"} Jul 07 18:45:35.764064 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:35.764028 2582 generic.go:356] "Generic (PLEG): container finished" podID="107cfa3d-ccec-412f-987b-10b2b0c2174f" containerID="b423d5f442eb13f702be386f0f1755ea8a38182618ab13b137c4ae2f66ae9ba9" exitCode=0 Jul 07 18:45:35.764430 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:35.764099 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" event={"ID":"107cfa3d-ccec-412f-987b-10b2b0c2174f","Type":"ContainerDied","Data":"b423d5f442eb13f702be386f0f1755ea8a38182618ab13b137c4ae2f66ae9ba9"} Jul 07 18:45:36.581500 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:36.581465 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:36.581707 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:36.581574 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:36.581707 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:36.581584 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srz9n" podUID="19e9d15b-2047-4bb5-846d-dd0ffe63d8cd" Jul 07 18:45:36.581707 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:36.581598 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:36.581707 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:36.581676 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nmhjf" podUID="2753fb64-3975-4312-96ea-9cf482c0081c" Jul 07 18:45:36.581935 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:36.581750 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4hlp" podUID="8af537b1-f55e-42ec-9269-1f55dc81a5ec" Jul 07 18:45:36.769098 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:36.769068 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" event={"ID":"107cfa3d-ccec-412f-987b-10b2b0c2174f","Type":"ContainerStarted","Data":"8c704e3aa7da84f8a97e702b4e6ea0a1482fbe7012e3a06fd37dc2ad370f2919"} Jul 07 18:45:36.794616 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:36.794561 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jtdd2" podStartSLOduration=10.3127479 podStartE2EDuration="38.794544719s" totalImagesPullingTime="28.481796819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:36.794255164 +0000 UTC m=+38.819373237" watchObservedRunningTime="2026-07-07 18:45:36.794544719 +0000 UTC m=+38.819662802" Jul 07 18:45:37.270144 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.270116 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-163.ec2.internal" event="NodeReady" Jul 07 18:45:37.270309 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.270237 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 18:45:37.320911 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.320879 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-snt25"] Jul 07 18:45:37.343930 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.343904 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2wzjs"] Jul 07 18:45:37.344096 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.344041 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.346259 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.346227 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 18:45:37.346392 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.346266 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pgd5v\"" Jul 07 18:45:37.346392 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.346320 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 18:45:37.368228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.368192 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-snt25"] Jul 07 18:45:37.368228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.368235 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2wzjs"] Jul 07 18:45:37.368416 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.368297 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2wzjs" Jul 07 18:45:37.370656 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.370619 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 18:45:37.370768 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.370665 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 18:45:37.370768 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.370718 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w2mm7\"" Jul 07 18:45:37.370918 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.370903 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 18:45:37.448212 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.448162 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.448212 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.448218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmv2j\" (UniqueName: \"kubernetes.io/projected/b9576f40-fc27-45a0-a17d-d19d7da6676b-kube-api-access-wmv2j\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.448442 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.448295 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9576f40-fc27-45a0-a17d-d19d7da6676b-config-volume\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.448442 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.448333 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9576f40-fc27-45a0-a17d-d19d7da6676b-tmp-dir\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.549412 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.549319 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll5qb\" (UniqueName: \"kubernetes.io/projected/45d31a00-8411-45f4-832d-12ddf08417f5-kube-api-access-ll5qb\") pod \"ingress-canary-2wzjs\" (UID: \"45d31a00-8411-45f4-832d-12ddf08417f5\") " pod="openshift-ingress-canary/ingress-canary-2wzjs" Jul 07 18:45:37.549412 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.549379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9576f40-fc27-45a0-a17d-d19d7da6676b-config-volume\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.549638 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.549422 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9576f40-fc27-45a0-a17d-d19d7da6676b-tmp-dir\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.549638 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.549479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.549638 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.549506 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45d31a00-8411-45f4-832d-12ddf08417f5-cert\") pod \"ingress-canary-2wzjs\" (UID: \"45d31a00-8411-45f4-832d-12ddf08417f5\") " pod="openshift-ingress-canary/ingress-canary-2wzjs" Jul 07 18:45:37.549638 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.549536 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmv2j\" (UniqueName: \"kubernetes.io/projected/b9576f40-fc27-45a0-a17d-d19d7da6676b-kube-api-access-wmv2j\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.549638 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:37.549609 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:45:37.552529 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:37.550028 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls podName:b9576f40-fc27-45a0-a17d-d19d7da6676b nodeName:}" failed. No retries permitted until 2026-07-07 18:45:38.049997292 +0000 UTC m=+40.075115351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls") pod "dns-default-snt25" (UID: "b9576f40-fc27-45a0-a17d-d19d7da6676b") : secret "dns-default-metrics-tls" not found Jul 07 18:45:37.552529 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.550339 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9576f40-fc27-45a0-a17d-d19d7da6676b-config-volume\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.561914 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.561893 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9576f40-fc27-45a0-a17d-d19d7da6676b-tmp-dir\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.565906 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.565885 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmv2j\" (UniqueName: \"kubernetes.io/projected/b9576f40-fc27-45a0-a17d-d19d7da6676b-kube-api-access-wmv2j\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:37.650639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.650608 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45d31a00-8411-45f4-832d-12ddf08417f5-cert\") pod \"ingress-canary-2wzjs\" (UID: \"45d31a00-8411-45f4-832d-12ddf08417f5\") " pod="openshift-ingress-canary/ingress-canary-2wzjs" Jul 07 18:45:37.650814 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.650681 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll5qb\" (UniqueName: \"kubernetes.io/projected/45d31a00-8411-45f4-832d-12ddf08417f5-kube-api-access-ll5qb\") pod \"ingress-canary-2wzjs\" (UID: \"45d31a00-8411-45f4-832d-12ddf08417f5\") " pod="openshift-ingress-canary/ingress-canary-2wzjs" Jul 07 18:45:37.653151 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.653122 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45d31a00-8411-45f4-832d-12ddf08417f5-cert\") pod \"ingress-canary-2wzjs\" (UID: \"45d31a00-8411-45f4-832d-12ddf08417f5\") " pod="openshift-ingress-canary/ingress-canary-2wzjs" Jul 07 18:45:37.660247 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.660219 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll5qb\" (UniqueName: \"kubernetes.io/projected/45d31a00-8411-45f4-832d-12ddf08417f5-kube-api-access-ll5qb\") pod \"ingress-canary-2wzjs\" (UID: \"45d31a00-8411-45f4-832d-12ddf08417f5\") " pod="openshift-ingress-canary/ingress-canary-2wzjs" Jul 07 18:45:37.677024 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.677000 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2wzjs" Jul 07 18:45:37.886470 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:37.886239 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2wzjs"] Jul 07 18:45:37.890016 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:45:37.889990 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d31a00_8411_45f4_832d_12ddf08417f5.slice/crio-09fc1d4b104d2a104d1fd6bf43eafbb9d7eb8807acb7ba0bfb0176b2d32130cd WatchSource:0}: Error finding container 09fc1d4b104d2a104d1fd6bf43eafbb9d7eb8807acb7ba0bfb0176b2d32130cd: Status 404 returned error can't find the container with id 09fc1d4b104d2a104d1fd6bf43eafbb9d7eb8807acb7ba0bfb0176b2d32130cd Jul 07 18:45:38.054841 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.054806 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:38.055006 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:38.054920 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:45:38.055006 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:38.054975 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls podName:b9576f40-fc27-45a0-a17d-d19d7da6676b nodeName:}" failed. No retries permitted until 2026-07-07 18:45:39.054959883 +0000 UTC m=+41.080077957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls") pod "dns-default-snt25" (UID: "b9576f40-fc27-45a0-a17d-d19d7da6676b") : secret "dns-default-metrics-tls" not found Jul 07 18:45:38.583173 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.583143 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:45:38.583379 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.583269 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:45:38.583669 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.583501 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:45:38.586933 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.586907 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 18:45:38.586933 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.586922 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 18:45:38.587102 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.586952 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vdqwt\"" Jul 07 18:45:38.587102 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.586963 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8gxh7\"" Jul 07 18:45:38.587102 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.586998 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 18:45:38.587102 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.586907 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 18:45:38.775727 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:38.775687 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2wzjs" event={"ID":"45d31a00-8411-45f4-832d-12ddf08417f5","Type":"ContainerStarted","Data":"09fc1d4b104d2a104d1fd6bf43eafbb9d7eb8807acb7ba0bfb0176b2d32130cd"} Jul 07 18:45:39.062307 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:39.062274 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:39.062737 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:39.062384 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:45:39.062737 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:39.062448 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls podName:b9576f40-fc27-45a0-a17d-d19d7da6676b nodeName:}" failed. No retries permitted until 2026-07-07 18:45:41.062434983 +0000 UTC m=+43.087553039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls") pod "dns-default-snt25" (UID: "b9576f40-fc27-45a0-a17d-d19d7da6676b") : secret "dns-default-metrics-tls" not found Jul 07 18:45:41.078456 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:41.078419 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:41.078853 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:41.078548 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:45:41.078853 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:41.078598 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls podName:b9576f40-fc27-45a0-a17d-d19d7da6676b nodeName:}" failed. No retries permitted until 2026-07-07 18:45:45.078585092 +0000 UTC m=+47.103703143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls") pod "dns-default-snt25" (UID: "b9576f40-fc27-45a0-a17d-d19d7da6676b") : secret "dns-default-metrics-tls" not found Jul 07 18:45:41.784891 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:41.784854 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2wzjs" event={"ID":"45d31a00-8411-45f4-832d-12ddf08417f5","Type":"ContainerStarted","Data":"175ea171508511d7455c563e86702bb52dd20a91d33f38d03c0065444f2be53e"} Jul 07 18:45:41.804289 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:41.804239 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2wzjs" podStartSLOduration=1.9088931169999999 podStartE2EDuration="4.804223696s" totalImagesPullingTime="2.895330579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:45:41.803680794 +0000 UTC m=+43.828798868" watchObservedRunningTime="2026-07-07 18:45:41.804223696 +0000 UTC m=+43.829341770" Jul 07 18:45:45.105549 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:45.105513 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:45.105966 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:45.105635 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:45:45.105966 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:45.105714 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls podName:b9576f40-fc27-45a0-a17d-d19d7da6676b nodeName:}" failed. No retries permitted until 2026-07-07 18:45:53.105698259 +0000 UTC m=+55.130816324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls") pod "dns-default-snt25" (UID: "b9576f40-fc27-45a0-a17d-d19d7da6676b") : secret "dns-default-metrics-tls" not found Jul 07 18:45:53.159182 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:45:53.158987 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:45:53.159755 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:53.159148 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:45:53.159755 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:45:53.159259 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls podName:b9576f40-fc27-45a0-a17d-d19d7da6676b nodeName:}" failed. No retries permitted until 2026-07-07 18:46:09.159244154 +0000 UTC m=+71.184362205 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls") pod "dns-default-snt25" (UID: "b9576f40-fc27-45a0-a17d-d19d7da6676b") : secret "dns-default-metrics-tls" not found Jul 07 18:46:01.761468 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:01.761433 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7x6bl" Jul 07 18:46:04.131593 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.131551 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:46:04.131593 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.131606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:46:04.134602 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.134581 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 18:46:04.134602 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.134597 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 18:46:04.142815 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:04.142793 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 18:46:04.142884 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:04.142861 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs podName:19e9d15b-2047-4bb5-846d-dd0ffe63d8cd nodeName:}" failed. No retries permitted until 2026-07-07 18:47:08.142843242 +0000 UTC m=+130.167961293 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs") pod "network-metrics-daemon-srz9n" (UID: "19e9d15b-2047-4bb5-846d-dd0ffe63d8cd") : secret "metrics-daemon-secret" not found Jul 07 18:46:04.146507 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.146488 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8af537b1-f55e-42ec-9269-1f55dc81a5ec-original-pull-secret\") pod \"global-pull-secret-syncer-b4hlp\" (UID: \"8af537b1-f55e-42ec-9269-1f55dc81a5ec\") " pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:46:04.232368 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.232331 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:46:04.234818 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.234797 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 18:46:04.237766 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.237746 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt"] Jul 07 18:46:04.245118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.245097 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 18:46:04.257021 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.256988 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l4b9\" (UniqueName: \"kubernetes.io/projected/2753fb64-3975-4312-96ea-9cf482c0081c-kube-api-access-8l4b9\") pod \"network-check-target-nmhjf\" (UID: \"2753fb64-3975-4312-96ea-9cf482c0081c\") " pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:46:04.272871 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.272841 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt"] Jul 07 18:46:04.272995 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.272931 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.276395 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.276375 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 18:46:04.277355 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.277335 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tpqd4\"" Jul 07 18:46:04.277450 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.277371 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 18:46:04.277450 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.277371 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 18:46:04.277559 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.277371 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 18:46:04.333174 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.333144 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/012c2fba-0533-43e7-b7c3-7d3476290739-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.333174 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.333187 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.333404 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.333207 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7285g\" (UniqueName: \"kubernetes.io/projected/012c2fba-0533-43e7-b7c3-7d3476290739-kube-api-access-7285g\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.398422 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.398339 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vdqwt\"" Jul 07 18:46:04.404103 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.404087 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4hlp" Jul 07 18:46:04.407230 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.407204 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:46:04.434359 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.434333 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/012c2fba-0533-43e7-b7c3-7d3476290739-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.434496 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.434376 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.434496 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.434396 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7285g\" (UniqueName: \"kubernetes.io/projected/012c2fba-0533-43e7-b7c3-7d3476290739-kube-api-access-7285g\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.434496 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:04.434482 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:04.434680 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:04.434547 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls podName:012c2fba-0533-43e7-b7c3-7d3476290739 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:04.934531565 +0000 UTC m=+66.959649616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-rv5jt" (UID: "012c2fba-0533-43e7-b7c3-7d3476290739") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:04.435072 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.435050 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/012c2fba-0533-43e7-b7c3-7d3476290739-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.445483 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.445454 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7285g\" (UniqueName: \"kubernetes.io/projected/012c2fba-0533-43e7-b7c3-7d3476290739-kube-api-access-7285g\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.552417 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.552377 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b4hlp"] Jul 07 18:46:04.556534 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:04.556495 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af537b1_f55e_42ec_9269_1f55dc81a5ec.slice/crio-8d22f6e190e26dbbc5b34e9c9a2de5f08b17ccb7a40f907544d949bfa717c652 WatchSource:0}: Error finding container 8d22f6e190e26dbbc5b34e9c9a2de5f08b17ccb7a40f907544d949bfa717c652: Status 404 returned error can't find the container with id 8d22f6e190e26dbbc5b34e9c9a2de5f08b17ccb7a40f907544d949bfa717c652 Jul 07 18:46:04.569811 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.569787 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nmhjf"] Jul 07 18:46:04.574369 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:04.574348 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2753fb64_3975_4312_96ea_9cf482c0081c.slice/crio-1d73bec267db6f883cc981cc9393ed00118ba0b598fa69ca19d09a113ea4307b WatchSource:0}: Error finding container 1d73bec267db6f883cc981cc9393ed00118ba0b598fa69ca19d09a113ea4307b: Status 404 returned error can't find the container with id 1d73bec267db6f883cc981cc9393ed00118ba0b598fa69ca19d09a113ea4307b Jul 07 18:46:04.838220 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.838188 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nmhjf" event={"ID":"2753fb64-3975-4312-96ea-9cf482c0081c","Type":"ContainerStarted","Data":"1d73bec267db6f883cc981cc9393ed00118ba0b598fa69ca19d09a113ea4307b"} Jul 07 18:46:04.839232 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.839209 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b4hlp" event={"ID":"8af537b1-f55e-42ec-9269-1f55dc81a5ec","Type":"ContainerStarted","Data":"8d22f6e190e26dbbc5b34e9c9a2de5f08b17ccb7a40f907544d949bfa717c652"} Jul 07 18:46:04.937845 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:04.937817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:04.938009 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:04.937959 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:04.938059 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:04.938023 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls podName:012c2fba-0533-43e7-b7c3-7d3476290739 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:05.938007933 +0000 UTC m=+67.963125984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-rv5jt" (UID: "012c2fba-0533-43e7-b7c3-7d3476290739") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:05.944857 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:05.944822 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:05.945275 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:05.944975 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:05.945275 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:05.945032 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls podName:012c2fba-0533-43e7-b7c3-7d3476290739 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:07.945017158 +0000 UTC m=+69.970135213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-rv5jt" (UID: "012c2fba-0533-43e7-b7c3-7d3476290739") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:07.957470 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.957424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:07.957973 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:07.957598 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:07.957973 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:07.957682 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls podName:012c2fba-0533-43e7-b7c3-7d3476290739 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:11.957666171 +0000 UTC m=+73.982784223 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-rv5jt" (UID: "012c2fba-0533-43e7-b7c3-7d3476290739") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:07.971519 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.971488 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cw9rd"] Jul 07 18:46:07.974022 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.974000 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:07.976230 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.976205 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 18:46:07.976580 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.976556 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 18:46:07.976713 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.976696 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 18:46:07.976828 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.976810 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 18:46:07.977525 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.977506 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-htdwq\"" Jul 07 18:46:07.986543 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:07.985865 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cw9rd"] Jul 07 18:46:08.058905 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.058865 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/095c0673-2f70-4197-98fa-194a67fb7715-data-volume\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.059059 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.058916 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np4kk\" (UniqueName: \"kubernetes.io/projected/095c0673-2f70-4197-98fa-194a67fb7715-kube-api-access-np4kk\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.059059 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.058971 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/095c0673-2f70-4197-98fa-194a67fb7715-crio-socket\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.059059 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.059023 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.059059 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.059047 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.159315 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.159281 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/095c0673-2f70-4197-98fa-194a67fb7715-data-volume\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.159315 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.159319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np4kk\" (UniqueName: \"kubernetes.io/projected/095c0673-2f70-4197-98fa-194a67fb7715-kube-api-access-np4kk\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.159532 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.159374 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/095c0673-2f70-4197-98fa-194a67fb7715-crio-socket\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.159532 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.159432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.159532 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.159460 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.159689 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:08.159557 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:08.159689 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:08.159614 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:08.659597676 +0000 UTC m=+70.684715727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:08.159789 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.159694 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/095c0673-2f70-4197-98fa-194a67fb7715-crio-socket\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.159789 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.159701 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/095c0673-2f70-4197-98fa-194a67fb7715-data-volume\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.159789 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:08.159751 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:08.159921 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:08.159799 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:08.659786879 +0000 UTC m=+70.684904930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : configmap "kube-rbac-proxy" not found Jul 07 18:46:08.168276 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.168251 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np4kk\" (UniqueName: \"kubernetes.io/projected/095c0673-2f70-4197-98fa-194a67fb7715-kube-api-access-np4kk\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.663046 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.662997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.663046 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:08.663049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:08.663279 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:08.663127 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:08.663279 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:08.663162 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:08.663279 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:08.663219 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:09.663196359 +0000 UTC m=+71.688314416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : configmap "kube-rbac-proxy" not found Jul 07 18:46:08.663279 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:08.663240 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:09.663230112 +0000 UTC m=+71.688348178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:09.165906 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:09.165863 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:46:09.166427 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:09.166030 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:46:09.166427 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:09.166107 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls podName:b9576f40-fc27-45a0-a17d-d19d7da6676b nodeName:}" failed. No retries permitted until 2026-07-07 18:46:41.166089073 +0000 UTC m=+103.191207124 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls") pod "dns-default-snt25" (UID: "b9576f40-fc27-45a0-a17d-d19d7da6676b") : secret "dns-default-metrics-tls" not found Jul 07 18:46:09.670183 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:09.670144 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:09.670183 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:09.670187 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:09.670436 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:09.670308 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:09.670436 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:09.670307 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:09.670436 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:09.670378 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:11.670360914 +0000 UTC m=+73.695478978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:09.670436 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:09.670400 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:11.670388504 +0000 UTC m=+73.695506555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : configmap "kube-rbac-proxy" not found Jul 07 18:46:09.747665 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:09.747607 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5wwvt_039885f1-e2ef-4c36-8fe1-c1b0a1a023ea/dns-node-resolver/0.log" Jul 07 18:46:10.547417 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:10.547390 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8lmc_0d3043a3-d10f-4fef-bbe1-7f9d18666671/node-ca/0.log" Jul 07 18:46:10.855783 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:10.855746 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b4hlp" event={"ID":"8af537b1-f55e-42ec-9269-1f55dc81a5ec","Type":"ContainerStarted","Data":"df76ad4bee7d1991ef2a3ac7be377eca2e08ee90dfc4536b5713d6b7047f81f1"} Jul 07 18:46:10.857205 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:10.857178 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nmhjf" event={"ID":"2753fb64-3975-4312-96ea-9cf482c0081c","Type":"ContainerStarted","Data":"ce68211e903d1baf2f7d9eee23c174feba3bd21687d5c8f37fa5eb824f5d5863"} Jul 07 18:46:10.857315 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:10.857296 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:46:10.871971 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:10.871926 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b4hlp" podStartSLOduration=67.434507568 podStartE2EDuration="1m12.871909379s" totalImagesPullingTime="5.437401811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:10.871331228 +0000 UTC m=+72.896449302" watchObservedRunningTime="2026-07-07 18:46:10.871909379 +0000 UTC m=+72.897027453" Jul 07 18:46:10.886552 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:10.886512 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nmhjf" podStartSLOduration=67.467895998 podStartE2EDuration="1m12.886497489s" totalImagesPullingTime="5.418601491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:10.885898022 +0000 UTC m=+72.911016096" watchObservedRunningTime="2026-07-07 18:46:10.886497489 +0000 UTC m=+72.911615561" Jul 07 18:46:11.351674 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:11.351624 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2wzjs_45d31a00-8411-45f4-832d-12ddf08417f5/serve-healthcheck-canary/0.log" Jul 07 18:46:11.684180 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:11.684083 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:11.684180 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:11.684133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:11.684576 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:11.684208 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:11.684576 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:11.684221 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:11.684576 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:11.684270 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:15.684257953 +0000 UTC m=+77.709376003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:11.684576 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:11.684282 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:15.684276669 +0000 UTC m=+77.709394720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : configmap "kube-rbac-proxy" not found Jul 07 18:46:11.985938 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:11.985852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:11.986111 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:11.985999 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:11.986111 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:11.986059 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls podName:012c2fba-0533-43e7-b7c3-7d3476290739 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:19.986042591 +0000 UTC m=+82.011160647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-rv5jt" (UID: "012c2fba-0533-43e7-b7c3-7d3476290739") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:14.254738 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.254702 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq"] Jul 07 18:46:14.257537 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.257515 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq" Jul 07 18:46:14.257740 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.257718 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq"] Jul 07 18:46:14.259344 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.259324 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.259816 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.259798 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:14.260490 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.260474 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:14.260576 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.260540 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-brmbw\"" Jul 07 18:46:14.261197 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.261181 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:14.261575 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.261558 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:14.261672 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.261558 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-92fl7\"" Jul 07 18:46:14.261753 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.261704 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 18:46:14.261753 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.261721 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 18:46:14.269029 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.268966 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq"] Jul 07 18:46:14.269901 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.269827 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq"] Jul 07 18:46:14.300223 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.300191 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32ea9797-0ffc-466c-82ec-4a6217043139-config\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.300392 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.300232 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6skdf\" (UniqueName: \"kubernetes.io/projected/33da68e1-9493-4e0d-acf2-8bcad8072fa5-kube-api-access-6skdf\") pod \"volume-data-source-validator-5d456fdbcd-t4cwq\" (UID: \"33da68e1-9493-4e0d-acf2-8bcad8072fa5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq" Jul 07 18:46:14.300392 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.300263 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4xld\" (UniqueName: \"kubernetes.io/projected/32ea9797-0ffc-466c-82ec-4a6217043139-kube-api-access-z4xld\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.300392 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.300289 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32ea9797-0ffc-466c-82ec-4a6217043139-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.363627 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.363591 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln"] Jul 07 18:46:14.365385 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.365369 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-wwpsl"] Jul 07 18:46:14.365500 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.365483 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.367992 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.367969 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.368755 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.368740 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 18:46:14.369227 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.369209 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:14.369362 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.369345 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 18:46:14.369474 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.369449 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-m926z\"" Jul 07 18:46:14.369599 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.369583 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:14.374447 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.374429 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:14.377538 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.377519 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 18:46:14.377632 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.377571 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 18:46:14.377632 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.377586 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 18:46:14.378305 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.378287 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6qtcr\"" Jul 07 18:46:14.393007 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.392987 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln"] Jul 07 18:46:14.393746 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.393729 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-wwpsl"] Jul 07 18:46:14.399047 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.399025 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 18:46:14.401338 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401317 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4c436eb0-0acc-4322-9ab0-3611167e3e47-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.401445 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401359 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32ea9797-0ffc-466c-82ec-4a6217043139-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.401445 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401389 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rqwx\" (UniqueName: \"kubernetes.io/projected/4c436eb0-0acc-4322-9ab0-3611167e3e47-kube-api-access-4rqwx\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.401445 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401409 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8383d520-8b64-45f6-b79d-a50b06f0e20c-config\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.401445 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401443 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4c436eb0-0acc-4322-9ab0-3611167e3e47-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.401667 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401460 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8383d520-8b64-45f6-b79d-a50b06f0e20c-serving-cert\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.401667 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32ea9797-0ffc-466c-82ec-4a6217043139-config\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.401667 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401629 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8383d520-8b64-45f6-b79d-a50b06f0e20c-trusted-ca\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.402023 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401692 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75j75\" (UniqueName: \"kubernetes.io/projected/8383d520-8b64-45f6-b79d-a50b06f0e20c-kube-api-access-75j75\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.402023 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6skdf\" (UniqueName: \"kubernetes.io/projected/33da68e1-9493-4e0d-acf2-8bcad8072fa5-kube-api-access-6skdf\") pod \"volume-data-source-validator-5d456fdbcd-t4cwq\" (UID: \"33da68e1-9493-4e0d-acf2-8bcad8072fa5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq" Jul 07 18:46:14.402023 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.401804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4xld\" (UniqueName: \"kubernetes.io/projected/32ea9797-0ffc-466c-82ec-4a6217043139-kube-api-access-z4xld\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.402132 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.402064 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32ea9797-0ffc-466c-82ec-4a6217043139-config\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.403937 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.403918 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32ea9797-0ffc-466c-82ec-4a6217043139-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.409929 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.409907 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4xld\" (UniqueName: \"kubernetes.io/projected/32ea9797-0ffc-466c-82ec-4a6217043139-kube-api-access-z4xld\") pod \"service-ca-operator-5d46d9ffd4-5vggq\" (UID: \"32ea9797-0ffc-466c-82ec-4a6217043139\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.410049 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.410033 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6skdf\" (UniqueName: \"kubernetes.io/projected/33da68e1-9493-4e0d-acf2-8bcad8072fa5-kube-api-access-6skdf\") pod \"volume-data-source-validator-5d456fdbcd-t4cwq\" (UID: \"33da68e1-9493-4e0d-acf2-8bcad8072fa5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq" Jul 07 18:46:14.502157 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.502114 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rqwx\" (UniqueName: \"kubernetes.io/projected/4c436eb0-0acc-4322-9ab0-3611167e3e47-kube-api-access-4rqwx\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.502157 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.502157 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8383d520-8b64-45f6-b79d-a50b06f0e20c-config\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.502420 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.502191 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4c436eb0-0acc-4322-9ab0-3611167e3e47-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.502420 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.502205 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8383d520-8b64-45f6-b79d-a50b06f0e20c-serving-cert\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.502420 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.502236 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8383d520-8b64-45f6-b79d-a50b06f0e20c-trusted-ca\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.502420 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.502255 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75j75\" (UniqueName: \"kubernetes.io/projected/8383d520-8b64-45f6-b79d-a50b06f0e20c-kube-api-access-75j75\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.502601 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.502486 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4c436eb0-0acc-4322-9ab0-3611167e3e47-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.503070 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.503045 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8383d520-8b64-45f6-b79d-a50b06f0e20c-config\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.503199 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.503181 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4c436eb0-0acc-4322-9ab0-3611167e3e47-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.503610 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.503592 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8383d520-8b64-45f6-b79d-a50b06f0e20c-trusted-ca\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.504721 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.504701 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4c436eb0-0acc-4322-9ab0-3611167e3e47-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.504832 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.504758 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8383d520-8b64-45f6-b79d-a50b06f0e20c-serving-cert\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.512472 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.512451 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rqwx\" (UniqueName: \"kubernetes.io/projected/4c436eb0-0acc-4322-9ab0-3611167e3e47-kube-api-access-4rqwx\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7tln\" (UID: \"4c436eb0-0acc-4322-9ab0-3611167e3e47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.512581 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.512554 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75j75\" (UniqueName: \"kubernetes.io/projected/8383d520-8b64-45f6-b79d-a50b06f0e20c-kube-api-access-75j75\") pod \"console-operator-68b757fb67-wwpsl\" (UID: \"8383d520-8b64-45f6-b79d-a50b06f0e20c\") " pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.566636 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.566601 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq" Jul 07 18:46:14.572547 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.572526 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" Jul 07 18:46:14.675996 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.675952 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" Jul 07 18:46:14.680147 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.680111 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:14.703298 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.703266 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq"] Jul 07 18:46:14.707285 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:14.707252 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33da68e1_9493_4e0d_acf2_8bcad8072fa5.slice/crio-c29842f8e6770ff996cf1880a8ffca36049203406224b0887c2349217d37b043 WatchSource:0}: Error finding container c29842f8e6770ff996cf1880a8ffca36049203406224b0887c2349217d37b043: Status 404 returned error can't find the container with id c29842f8e6770ff996cf1880a8ffca36049203406224b0887c2349217d37b043 Jul 07 18:46:14.714953 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.714897 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq"] Jul 07 18:46:14.718420 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:14.718394 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ea9797_0ffc_466c_82ec_4a6217043139.slice/crio-cb9067ebc8a7a8f6e39699712f2db4ddc87d7de0d838d1dd6dbd2b95f0641e05 WatchSource:0}: Error finding container cb9067ebc8a7a8f6e39699712f2db4ddc87d7de0d838d1dd6dbd2b95f0641e05: Status 404 returned error can't find the container with id cb9067ebc8a7a8f6e39699712f2db4ddc87d7de0d838d1dd6dbd2b95f0641e05 Jul 07 18:46:14.817914 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.817883 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-wwpsl"] Jul 07 18:46:14.820696 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:14.820668 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8383d520_8b64_45f6_b79d_a50b06f0e20c.slice/crio-d1acb77a3143f0d34783fbb586ead10aba63179147ce9aa3ed9908433d30195c WatchSource:0}: Error finding container d1acb77a3143f0d34783fbb586ead10aba63179147ce9aa3ed9908433d30195c: Status 404 returned error can't find the container with id d1acb77a3143f0d34783fbb586ead10aba63179147ce9aa3ed9908433d30195c Jul 07 18:46:14.832294 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.832271 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln"] Jul 07 18:46:14.834972 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:14.834952 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c436eb0_0acc_4322_9ab0_3611167e3e47.slice/crio-d78fbd4e85f5135fe5fd6d034776114586fe56fa2838a62c37aa7547368d0165 WatchSource:0}: Error finding container d78fbd4e85f5135fe5fd6d034776114586fe56fa2838a62c37aa7547368d0165: Status 404 returned error can't find the container with id d78fbd4e85f5135fe5fd6d034776114586fe56fa2838a62c37aa7547368d0165 Jul 07 18:46:14.868467 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.868438 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" event={"ID":"8383d520-8b64-45f6-b79d-a50b06f0e20c","Type":"ContainerStarted","Data":"d1acb77a3143f0d34783fbb586ead10aba63179147ce9aa3ed9908433d30195c"} Jul 07 18:46:14.869469 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.869445 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq" event={"ID":"33da68e1-9493-4e0d-acf2-8bcad8072fa5","Type":"ContainerStarted","Data":"c29842f8e6770ff996cf1880a8ffca36049203406224b0887c2349217d37b043"} Jul 07 18:46:14.870441 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.870418 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" event={"ID":"32ea9797-0ffc-466c-82ec-4a6217043139","Type":"ContainerStarted","Data":"cb9067ebc8a7a8f6e39699712f2db4ddc87d7de0d838d1dd6dbd2b95f0641e05"} Jul 07 18:46:14.871292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:14.871275 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" event={"ID":"4c436eb0-0acc-4322-9ab0-3611167e3e47","Type":"ContainerStarted","Data":"d78fbd4e85f5135fe5fd6d034776114586fe56fa2838a62c37aa7547368d0165"} Jul 07 18:46:15.712889 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:15.712853 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:15.713328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:15.712905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:15.713328 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:15.712993 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:46:15.713328 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:15.713070 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:23.713050055 +0000 UTC m=+85.738168111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : configmap "kube-rbac-proxy" not found Jul 07 18:46:15.713328 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:15.713102 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:15.713328 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:15.713152 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:23.713138823 +0000 UTC m=+85.738256874 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:16.879292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:16.879253 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq" event={"ID":"33da68e1-9493-4e0d-acf2-8bcad8072fa5","Type":"ContainerStarted","Data":"e3ddffd2d9a43a84a6f0c5986dcc29ede5b60cac0119ede1948012d71c20e69a"} Jul 07 18:46:18.886670 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.886604 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" event={"ID":"4c436eb0-0acc-4322-9ab0-3611167e3e47","Type":"ContainerStarted","Data":"3eb0e30a4edcdd778feafb8ddd70684e25ab1ee29bc91bbb0a6a1d52aae41c69"} Jul 07 18:46:18.888237 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.888213 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/0.log" Jul 07 18:46:18.888346 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.888252 2582 generic.go:356] "Generic (PLEG): container finished" podID="8383d520-8b64-45f6-b79d-a50b06f0e20c" containerID="d1e220961c55622d18b613856971a55c27a58c0d5183404c2b3bc9b11c217e12" exitCode=255 Jul 07 18:46:18.888411 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.888338 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" event={"ID":"8383d520-8b64-45f6-b79d-a50b06f0e20c","Type":"ContainerDied","Data":"d1e220961c55622d18b613856971a55c27a58c0d5183404c2b3bc9b11c217e12"} Jul 07 18:46:18.888542 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.888522 2582 scope.go:117] "RemoveContainer" containerID="d1e220961c55622d18b613856971a55c27a58c0d5183404c2b3bc9b11c217e12" Jul 07 18:46:18.889858 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.889830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" event={"ID":"32ea9797-0ffc-466c-82ec-4a6217043139","Type":"ContainerStarted","Data":"b6af83f2b9f3f4de42fd0f331b734e22e09636bcfafbae895fd0cc87fdd0c606"} Jul 07 18:46:18.905026 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.904980 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-t4cwq" podStartSLOduration=3.727760864 podStartE2EDuration="4.904960619s" totalImagesPullingTime="1.177199755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:16.896791173 +0000 UTC m=+78.921909276" watchObservedRunningTime="2026-07-07 18:46:18.904960619 +0000 UTC m=+80.930078692" Jul 07 18:46:18.905150 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.905077 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" podStartSLOduration=1.533143249 podStartE2EDuration="4.905069904s" totalImagesPullingTime="3.371926655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:18.904394726 +0000 UTC m=+80.929512796" watchObservedRunningTime="2026-07-07 18:46:18.905069904 +0000 UTC m=+80.930187977" Jul 07 18:46:18.943207 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:18.942994 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" podStartSLOduration=1.450958107 podStartE2EDuration="4.942974892s" totalImagesPullingTime="3.492016785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:18.942378812 +0000 UTC m=+80.967496887" watchObservedRunningTime="2026-07-07 18:46:18.942974892 +0000 UTC m=+80.968092965" Jul 07 18:46:19.894975 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:19.894948 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/1.log" Jul 07 18:46:19.895530 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:19.895319 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/0.log" Jul 07 18:46:19.895530 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:19.895363 2582 generic.go:356] "Generic (PLEG): container finished" podID="8383d520-8b64-45f6-b79d-a50b06f0e20c" containerID="14dfa1d0c8ae27481f226bac81d94904161c2f1a32f3a913c5ae8115575e3d73" exitCode=255 Jul 07 18:46:19.895530 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:19.895429 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" event={"ID":"8383d520-8b64-45f6-b79d-a50b06f0e20c","Type":"ContainerDied","Data":"14dfa1d0c8ae27481f226bac81d94904161c2f1a32f3a913c5ae8115575e3d73"} Jul 07 18:46:19.895530 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:19.895490 2582 scope.go:117] "RemoveContainer" containerID="d1e220961c55622d18b613856971a55c27a58c0d5183404c2b3bc9b11c217e12" Jul 07 18:46:19.895873 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:19.895850 2582 scope.go:117] "RemoveContainer" containerID="14dfa1d0c8ae27481f226bac81d94904161c2f1a32f3a913c5ae8115575e3d73" Jul 07 18:46:19.896028 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:19.896010 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-wwpsl_openshift-console-operator(8383d520-8b64-45f6-b79d-a50b06f0e20c)\"" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" podUID="8383d520-8b64-45f6-b79d-a50b06f0e20c" Jul 07 18:46:20.047084 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.047046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:20.047259 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:20.047187 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:20.047307 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:20.047264 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls podName:012c2fba-0533-43e7-b7c3-7d3476290739 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:36.047247881 +0000 UTC m=+98.072365932 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-rv5jt" (UID: "012c2fba-0533-43e7-b7c3-7d3476290739") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:46:20.368757 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.368722 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh"] Jul 07 18:46:20.371473 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.371450 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" Jul 07 18:46:20.374063 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.374043 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 18:46:20.374200 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.374184 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-469jv\"" Jul 07 18:46:20.374544 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.374511 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 18:46:20.382433 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.382412 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh"] Jul 07 18:46:20.450227 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.450194 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29bfv\" (UniqueName: \"kubernetes.io/projected/275646cf-6982-42dc-838f-5bb27fb8ab45-kube-api-access-29bfv\") pod \"migrator-d57d58cc9-7vwsh\" (UID: \"275646cf-6982-42dc-838f-5bb27fb8ab45\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" Jul 07 18:46:20.551204 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.551167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29bfv\" (UniqueName: \"kubernetes.io/projected/275646cf-6982-42dc-838f-5bb27fb8ab45-kube-api-access-29bfv\") pod \"migrator-d57d58cc9-7vwsh\" (UID: \"275646cf-6982-42dc-838f-5bb27fb8ab45\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" Jul 07 18:46:20.561524 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.561496 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29bfv\" (UniqueName: \"kubernetes.io/projected/275646cf-6982-42dc-838f-5bb27fb8ab45-kube-api-access-29bfv\") pod \"migrator-d57d58cc9-7vwsh\" (UID: \"275646cf-6982-42dc-838f-5bb27fb8ab45\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" Jul 07 18:46:20.680577 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.680485 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" Jul 07 18:46:20.803108 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.803079 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh"] Jul 07 18:46:20.805697 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:20.805662 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275646cf_6982_42dc_838f_5bb27fb8ab45.slice/crio-4cca34b82fa3c497f1afc9c813537ad0b6deddd6d8bd7433e135b4929ec0abb8 WatchSource:0}: Error finding container 4cca34b82fa3c497f1afc9c813537ad0b6deddd6d8bd7433e135b4929ec0abb8: Status 404 returned error can't find the container with id 4cca34b82fa3c497f1afc9c813537ad0b6deddd6d8bd7433e135b4929ec0abb8 Jul 07 18:46:20.902838 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.902810 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/1.log" Jul 07 18:46:20.903286 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.903157 2582 scope.go:117] "RemoveContainer" containerID="14dfa1d0c8ae27481f226bac81d94904161c2f1a32f3a913c5ae8115575e3d73" Jul 07 18:46:20.903345 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:20.903327 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-wwpsl_openshift-console-operator(8383d520-8b64-45f6-b79d-a50b06f0e20c)\"" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" podUID="8383d520-8b64-45f6-b79d-a50b06f0e20c" Jul 07 18:46:20.904037 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:20.904016 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" event={"ID":"275646cf-6982-42dc-838f-5bb27fb8ab45","Type":"ContainerStarted","Data":"4cca34b82fa3c497f1afc9c813537ad0b6deddd6d8bd7433e135b4929ec0abb8"} Jul 07 18:46:21.964332 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:21.964300 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fzdtm"] Jul 07 18:46:21.966305 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:21.966289 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:21.968586 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:21.968565 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 18:46:21.968714 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:21.968564 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 18:46:21.969385 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:21.969360 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fgsk2\"" Jul 07 18:46:21.969505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:21.969404 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 18:46:21.969505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:21.969435 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 18:46:21.975155 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:21.975132 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fzdtm"] Jul 07 18:46:22.064191 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.064147 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-signing-cabundle\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.064365 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.064280 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6mjl\" (UniqueName: \"kubernetes.io/projected/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-kube-api-access-c6mjl\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.064365 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.064306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-signing-key\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.164867 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.164818 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6mjl\" (UniqueName: \"kubernetes.io/projected/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-kube-api-access-c6mjl\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.164867 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.164872 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-signing-key\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.165165 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.164928 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-signing-cabundle\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.165759 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.165732 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-signing-cabundle\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.168068 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.168040 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-signing-key\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.173872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.173847 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6mjl\" (UniqueName: \"kubernetes.io/projected/8ec3a5c2-251c-493f-a1e7-de7faf7443b7-kube-api-access-c6mjl\") pod \"service-ca-758f599c5-fzdtm\" (UID: \"8ec3a5c2-251c-493f-a1e7-de7faf7443b7\") " pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.275466 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.275380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-fzdtm" Jul 07 18:46:22.551205 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.551182 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fzdtm"] Jul 07 18:46:22.553958 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:22.553935 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec3a5c2_251c_493f_a1e7_de7faf7443b7.slice/crio-75683c47f2e3e4ea1214ecda8f90a94ead35f1ed142736157e9872f67792fee2 WatchSource:0}: Error finding container 75683c47f2e3e4ea1214ecda8f90a94ead35f1ed142736157e9872f67792fee2: Status 404 returned error can't find the container with id 75683c47f2e3e4ea1214ecda8f90a94ead35f1ed142736157e9872f67792fee2 Jul 07 18:46:22.913031 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.912998 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" event={"ID":"275646cf-6982-42dc-838f-5bb27fb8ab45","Type":"ContainerStarted","Data":"7224454184ce03eb0ac4b7678b8f5528e91472f2f103552c0c65c45f02de4131"} Jul 07 18:46:22.913205 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.913064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" event={"ID":"275646cf-6982-42dc-838f-5bb27fb8ab45","Type":"ContainerStarted","Data":"d3d2d8cd510bf45e4c7eb529666eb99f1dbe4221745f2676240436279aed6b4c"} Jul 07 18:46:22.914422 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.914400 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-fzdtm" event={"ID":"8ec3a5c2-251c-493f-a1e7-de7faf7443b7","Type":"ContainerStarted","Data":"c06157d7de0638757532f8cf1d0c5bf267a76c8d79d5f043816dd9ef1644dc7f"} Jul 07 18:46:22.914515 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.914434 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-fzdtm" event={"ID":"8ec3a5c2-251c-493f-a1e7-de7faf7443b7","Type":"ContainerStarted","Data":"75683c47f2e3e4ea1214ecda8f90a94ead35f1ed142736157e9872f67792fee2"} Jul 07 18:46:22.929839 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.929759 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-7vwsh" podStartSLOduration=1.289675791 podStartE2EDuration="2.929743282s" totalImagesPullingTime="1.640067491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:22.929691567 +0000 UTC m=+84.954809663" watchObservedRunningTime="2026-07-07 18:46:22.929743282 +0000 UTC m=+84.954861356" Jul 07 18:46:22.947547 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:22.947508 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-fzdtm" podStartSLOduration=1.947494663 podStartE2EDuration="1.947494663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:22.947009029 +0000 UTC m=+84.972127103" watchObservedRunningTime="2026-07-07 18:46:22.947494663 +0000 UTC m=+84.972612736" Jul 07 18:46:23.777522 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:23.777487 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:23.777522 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:23.777530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:23.778084 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:23.777722 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:46:23.778084 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:23.777791 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls podName:095c0673-2f70-4197-98fa-194a67fb7715 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:39.777770812 +0000 UTC m=+101.802888864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cw9rd" (UID: "095c0673-2f70-4197-98fa-194a67fb7715") : secret "insights-runtime-extractor-tls" not found Jul 07 18:46:23.778568 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:23.778539 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/095c0673-2f70-4197-98fa-194a67fb7715-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:24.681091 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:24.681055 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:24.681256 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:24.681110 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:24.681421 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:24.681410 2582 scope.go:117] "RemoveContainer" containerID="14dfa1d0c8ae27481f226bac81d94904161c2f1a32f3a913c5ae8115575e3d73" Jul 07 18:46:24.681560 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:24.681542 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-wwpsl_openshift-console-operator(8383d520-8b64-45f6-b79d-a50b06f0e20c)\"" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" podUID="8383d520-8b64-45f6-b79d-a50b06f0e20c" Jul 07 18:46:36.079416 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.079375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:36.081912 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.081892 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/012c2fba-0533-43e7-b7c3-7d3476290739-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-rv5jt\" (UID: \"012c2fba-0533-43e7-b7c3-7d3476290739\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:36.381437 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.381338 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" Jul 07 18:46:36.510399 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.510365 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt"] Jul 07 18:46:36.513181 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:36.513152 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012c2fba_0533_43e7_b7c3_7d3476290739.slice/crio-0b9d3ef12259cf8e5b7bbaced81a4317a2a7cbaf2d70d79f0c80fd059d4bb226 WatchSource:0}: Error finding container 0b9d3ef12259cf8e5b7bbaced81a4317a2a7cbaf2d70d79f0c80fd059d4bb226: Status 404 returned error can't find the container with id 0b9d3ef12259cf8e5b7bbaced81a4317a2a7cbaf2d70d79f0c80fd059d4bb226 Jul 07 18:46:36.579468 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.579434 2582 scope.go:117] "RemoveContainer" containerID="14dfa1d0c8ae27481f226bac81d94904161c2f1a32f3a913c5ae8115575e3d73" Jul 07 18:46:36.957703 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.957584 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" event={"ID":"012c2fba-0533-43e7-b7c3-7d3476290739","Type":"ContainerStarted","Data":"0b9d3ef12259cf8e5b7bbaced81a4317a2a7cbaf2d70d79f0c80fd059d4bb226"} Jul 07 18:46:36.959013 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.958990 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:46:36.959390 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.959376 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/1.log" Jul 07 18:46:36.959449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.959408 2582 generic.go:356] "Generic (PLEG): container finished" podID="8383d520-8b64-45f6-b79d-a50b06f0e20c" containerID="a440973c07d5e9feadd5dfc73c9e5b0a92ae959de6534cbac07a59aaa1fd6705" exitCode=255 Jul 07 18:46:36.959449 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.959436 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" event={"ID":"8383d520-8b64-45f6-b79d-a50b06f0e20c","Type":"ContainerDied","Data":"a440973c07d5e9feadd5dfc73c9e5b0a92ae959de6534cbac07a59aaa1fd6705"} Jul 07 18:46:36.959518 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.959470 2582 scope.go:117] "RemoveContainer" containerID="14dfa1d0c8ae27481f226bac81d94904161c2f1a32f3a913c5ae8115575e3d73" Jul 07 18:46:36.959803 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:36.959785 2582 scope.go:117] "RemoveContainer" containerID="a440973c07d5e9feadd5dfc73c9e5b0a92ae959de6534cbac07a59aaa1fd6705" Jul 07 18:46:36.959987 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:36.959968 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-wwpsl_openshift-console-operator(8383d520-8b64-45f6-b79d-a50b06f0e20c)\"" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" podUID="8383d520-8b64-45f6-b79d-a50b06f0e20c" Jul 07 18:46:37.964533 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:37.964502 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:46:38.969510 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:38.969472 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" event={"ID":"012c2fba-0533-43e7-b7c3-7d3476290739","Type":"ContainerStarted","Data":"4fb23e34eae36ce67c69e2fd19a38239f8091a2cb401f2c6d2d2d0ce0344fa0f"} Jul 07 18:46:38.986993 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:38.986946 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-rv5jt" podStartSLOduration=32.956984243 podStartE2EDuration="34.9869303s" totalImagesPullingTime="2.029946057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:38.985955413 +0000 UTC m=+101.011073486" watchObservedRunningTime="2026-07-07 18:46:38.9869303 +0000 UTC m=+101.012048373" Jul 07 18:46:39.806404 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:39.806362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:39.808800 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:39.808777 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/095c0673-2f70-4197-98fa-194a67fb7715-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cw9rd\" (UID: \"095c0673-2f70-4197-98fa-194a67fb7715\") " pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:40.084054 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:40.084016 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cw9rd" Jul 07 18:46:40.235950 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:40.235920 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cw9rd"] Jul 07 18:46:40.239210 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:40.239173 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095c0673_2f70_4197_98fa_194a67fb7715.slice/crio-858715a5dd4f11665a9d97917c111e4fc718ec1d590ada95c59ad7c0dc091b73 WatchSource:0}: Error finding container 858715a5dd4f11665a9d97917c111e4fc718ec1d590ada95c59ad7c0dc091b73: Status 404 returned error can't find the container with id 858715a5dd4f11665a9d97917c111e4fc718ec1d590ada95c59ad7c0dc091b73 Jul 07 18:46:40.978579 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:40.978489 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cw9rd" event={"ID":"095c0673-2f70-4197-98fa-194a67fb7715","Type":"ContainerStarted","Data":"858715a5dd4f11665a9d97917c111e4fc718ec1d590ada95c59ad7c0dc091b73"} Jul 07 18:46:41.221140 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:41.221101 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:46:41.224094 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:41.224045 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9576f40-fc27-45a0-a17d-d19d7da6676b-metrics-tls\") pod \"dns-default-snt25\" (UID: \"b9576f40-fc27-45a0-a17d-d19d7da6676b\") " pod="openshift-dns/dns-default-snt25" Jul 07 18:46:41.255545 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:41.255473 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pgd5v\"" Jul 07 18:46:41.263703 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:41.263676 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-snt25" Jul 07 18:46:41.409729 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:41.409682 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-snt25"] Jul 07 18:46:41.413331 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:41.413298 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9576f40_fc27_45a0_a17d_d19d7da6676b.slice/crio-a047b63c7b2d9c74369ab54d9f1d77989c9fc7ef7e7bd61cb41ce1bf326c235d WatchSource:0}: Error finding container a047b63c7b2d9c74369ab54d9f1d77989c9fc7ef7e7bd61cb41ce1bf326c235d: Status 404 returned error can't find the container with id a047b63c7b2d9c74369ab54d9f1d77989c9fc7ef7e7bd61cb41ce1bf326c235d Jul 07 18:46:41.861811 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:41.861785 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nmhjf" Jul 07 18:46:41.983703 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:41.983664 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-snt25" event={"ID":"b9576f40-fc27-45a0-a17d-d19d7da6676b","Type":"ContainerStarted","Data":"a047b63c7b2d9c74369ab54d9f1d77989c9fc7ef7e7bd61cb41ce1bf326c235d"} Jul 07 18:46:44.681080 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:44.681044 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:44.681448 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:44.681099 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:44.681448 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:44.681393 2582 scope.go:117] "RemoveContainer" containerID="a440973c07d5e9feadd5dfc73c9e5b0a92ae959de6534cbac07a59aaa1fd6705" Jul 07 18:46:44.681551 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:44.681532 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-wwpsl_openshift-console-operator(8383d520-8b64-45f6-b79d-a50b06f0e20c)\"" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" podUID="8383d520-8b64-45f6-b79d-a50b06f0e20c" Jul 07 18:46:45.280049 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.280015 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk"] Jul 07 18:46:45.307417 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.307288 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk"] Jul 07 18:46:45.307417 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.307402 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" Jul 07 18:46:45.310551 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.310524 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 18:46:45.311275 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.311252 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vqj4l\"" Jul 07 18:46:45.311381 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.311255 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 18:46:45.386614 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.386594 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9"] Jul 07 18:46:45.402915 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.402870 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" Jul 07 18:46:45.433815 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.433784 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9"] Jul 07 18:46:45.434911 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.434894 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qb8tf\"" Jul 07 18:46:45.452846 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.452816 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 18:46:45.457579 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.457557 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/acc883c3-9957-47d7-adef-481a0399c18a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wbqxk\" (UID: \"acc883c3-9957-47d7-adef-481a0399c18a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" Jul 07 18:46:45.457712 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.457590 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acc883c3-9957-47d7-adef-481a0399c18a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wbqxk\" (UID: \"acc883c3-9957-47d7-adef-481a0399c18a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" Jul 07 18:46:45.558177 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.558153 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/acc883c3-9957-47d7-adef-481a0399c18a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wbqxk\" (UID: \"acc883c3-9957-47d7-adef-481a0399c18a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" Jul 07 18:46:45.558262 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.558191 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acc883c3-9957-47d7-adef-481a0399c18a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wbqxk\" (UID: \"acc883c3-9957-47d7-adef-481a0399c18a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" Jul 07 18:46:45.558262 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.558239 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e51a18a3-d221-4a7c-aa76-3126282eb043-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rrvf9\" (UID: \"e51a18a3-d221-4a7c-aa76-3126282eb043\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" Jul 07 18:46:45.559062 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.558952 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/acc883c3-9957-47d7-adef-481a0399c18a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wbqxk\" (UID: \"acc883c3-9957-47d7-adef-481a0399c18a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" Jul 07 18:46:45.560599 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.560578 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/acc883c3-9957-47d7-adef-481a0399c18a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wbqxk\" (UID: \"acc883c3-9957-47d7-adef-481a0399c18a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" Jul 07 18:46:45.617280 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.617256 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" Jul 07 18:46:45.659771 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.659737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e51a18a3-d221-4a7c-aa76-3126282eb043-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rrvf9\" (UID: \"e51a18a3-d221-4a7c-aa76-3126282eb043\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" Jul 07 18:46:45.662400 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.662372 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e51a18a3-d221-4a7c-aa76-3126282eb043-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-rrvf9\" (UID: \"e51a18a3-d221-4a7c-aa76-3126282eb043\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" Jul 07 18:46:45.713003 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.712971 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" Jul 07 18:46:45.753968 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.753874 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk"] Jul 07 18:46:45.757351 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:45.757326 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc883c3_9957_47d7_adef_481a0399c18a.slice/crio-e95c2f38285d4c9c83b1da57570a936d3a0c46486e6b315ca27f8f7b88813280 WatchSource:0}: Error finding container e95c2f38285d4c9c83b1da57570a936d3a0c46486e6b315ca27f8f7b88813280: Status 404 returned error can't find the container with id e95c2f38285d4c9c83b1da57570a936d3a0c46486e6b315ca27f8f7b88813280 Jul 07 18:46:45.857679 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.855070 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9"] Jul 07 18:46:45.916325 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:45.916286 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51a18a3_d221_4a7c_aa76_3126282eb043.slice/crio-cdf73b14aeb45bff73df1265110dc8f7a528f47353371272c7e3234127037b0b WatchSource:0}: Error finding container cdf73b14aeb45bff73df1265110dc8f7a528f47353371272c7e3234127037b0b: Status 404 returned error can't find the container with id cdf73b14aeb45bff73df1265110dc8f7a528f47353371272c7e3234127037b0b Jul 07 18:46:45.998883 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.998852 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-snt25" event={"ID":"b9576f40-fc27-45a0-a17d-d19d7da6676b","Type":"ContainerStarted","Data":"20b65c3e963099aabea902809f4d87c7e7855a4a835b5a7324dc86934c817d65"} Jul 07 18:46:45.998977 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.998915 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-snt25" event={"ID":"b9576f40-fc27-45a0-a17d-d19d7da6676b","Type":"ContainerStarted","Data":"54b076c408e90c17744a7f90cf854d3ff0584c9f42a69adcdce79c6568b2e7a7"} Jul 07 18:46:45.999044 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:45.999000 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-snt25" Jul 07 18:46:46.000233 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:46.000211 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" event={"ID":"e51a18a3-d221-4a7c-aa76-3126282eb043","Type":"ContainerStarted","Data":"cdf73b14aeb45bff73df1265110dc8f7a528f47353371272c7e3234127037b0b"} Jul 07 18:46:46.002034 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:46.002013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cw9rd" event={"ID":"095c0673-2f70-4197-98fa-194a67fb7715","Type":"ContainerStarted","Data":"880eb595c2458977162e6dee12a691c555da77751095fbbc3eb1fe2abe6c8120"} Jul 07 18:46:46.002106 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:46.002064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cw9rd" event={"ID":"095c0673-2f70-4197-98fa-194a67fb7715","Type":"ContainerStarted","Data":"910d6bd171e8dcae7ae38bb0e4e9ba824c9421623b2cf7a6954c4be96cff5113"} Jul 07 18:46:46.003125 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:46.003105 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" event={"ID":"acc883c3-9957-47d7-adef-481a0399c18a","Type":"ContainerStarted","Data":"e95c2f38285d4c9c83b1da57570a936d3a0c46486e6b315ca27f8f7b88813280"} Jul 07 18:46:48.014200 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:48.014159 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" event={"ID":"e51a18a3-d221-4a7c-aa76-3126282eb043","Type":"ContainerStarted","Data":"6aca2c4195ee552db9b4555fc7cf5a1ec49f8cf04acb6cfdd26dfffa284865fb"} Jul 07 18:46:48.014773 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:48.014232 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" Jul 07 18:46:48.016495 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:48.016469 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" event={"ID":"acc883c3-9957-47d7-adef-481a0399c18a","Type":"ContainerStarted","Data":"95667d5e29af48fc7a51cc715eb4bda2677eec3b9a341698349b8a98d92587b3"} Jul 07 18:46:48.020199 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:48.020178 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" Jul 07 18:46:48.030491 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:48.030440 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-snt25" podStartSLOduration=66.971069944 podStartE2EDuration="1m11.030423336s" totalImagesPullingTime="4.059353392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:46.022022243 +0000 UTC m=+108.047140327" watchObservedRunningTime="2026-07-07 18:46:48.030423336 +0000 UTC m=+110.055541409" Jul 07 18:46:48.049166 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:48.049113 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-rrvf9" podStartSLOduration=1.6786455839999999 podStartE2EDuration="3.049096538s" totalImagesPullingTime="1.370450954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:48.031088724 +0000 UTC m=+110.056206799" watchObservedRunningTime="2026-07-07 18:46:48.049096538 +0000 UTC m=+110.074214611" Jul 07 18:46:48.065361 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:48.064925 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wbqxk" podStartSLOduration=1.537611644 podStartE2EDuration="3.064902703s" totalImagesPullingTime="1.527291059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:48.064104095 +0000 UTC m=+110.089222167" watchObservedRunningTime="2026-07-07 18:46:48.064902703 +0000 UTC m=+110.090020780" Jul 07 18:46:49.025498 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:49.025398 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cw9rd" event={"ID":"095c0673-2f70-4197-98fa-194a67fb7715","Type":"ContainerStarted","Data":"13fbe9420adf75167857ae0fdc924db45ca6b48736a9faedefd07f23c900b088"} Jul 07 18:46:49.045720 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:49.045673 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cw9rd" podStartSLOduration=33.69487336 podStartE2EDuration="42.045631181s" totalImagesPullingTime="8.350757821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:49.044447838 +0000 UTC m=+111.069565921" watchObservedRunningTime="2026-07-07 18:46:49.045631181 +0000 UTC m=+111.070749262" Jul 07 18:46:54.537395 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.537361 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m4ndg"] Jul 07 18:46:54.542442 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.542421 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt"] Jul 07 18:46:54.542597 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.542577 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.545083 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.544957 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 18:46:54.545083 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.544969 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd"] Jul 07 18:46:54.545083 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.545068 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.545284 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.545096 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 18:46:54.545284 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.544957 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 18:46:54.545742 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.545721 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t5r4f\"" Jul 07 18:46:54.546051 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.546037 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 18:46:54.547509 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.547490 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.547695 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.547680 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6kddl\"" Jul 07 18:46:54.547780 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.547757 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 18:46:54.547930 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.547914 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 18:46:54.549828 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.549812 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 18:46:54.549928 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.549823 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 18:46:54.549928 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.549823 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vqfzz\"" Jul 07 18:46:54.550038 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.549874 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 18:46:54.555541 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.555522 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt"] Jul 07 18:46:54.556268 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.556248 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd"] Jul 07 18:46:54.623115 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623084 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-tls\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623115 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623120 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-root\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623362 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623144 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.623362 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623197 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-textfile\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623362 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623216 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.623362 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbjqd\" (UniqueName: \"kubernetes.io/projected/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-api-access-sbjqd\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.623362 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623318 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2dq2\" (UniqueName: \"kubernetes.io/projected/209a5a3a-a456-41ae-b816-c430bcbb51ce-kube-api-access-m2dq2\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.623362 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psgwz\" (UniqueName: \"kubernetes.io/projected/896c42fc-177e-4e20-aa39-efcb2c534269-kube-api-access-psgwz\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623372 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-sys\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623395 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-wtmp\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623432 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-accelerators-collector-config\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623468 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/209a5a3a-a456-41ae-b816-c430bcbb51ce-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623494 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/896c42fc-177e-4e20-aa39-efcb2c534269-metrics-client-ca\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/209a5a3a-a456-41ae-b816-c430bcbb51ce-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623589 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623624 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.623747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.623712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/209a5a3a-a456-41ae-b816-c430bcbb51ce-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.724669 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724617 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-tls\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.724872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-root\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.724872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.724872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724751 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-textfile\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.724872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724779 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.724872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724803 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbjqd\" (UniqueName: \"kubernetes.io/projected/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-api-access-sbjqd\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.724872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2dq2\" (UniqueName: \"kubernetes.io/projected/209a5a3a-a456-41ae-b816-c430bcbb51ce-kube-api-access-m2dq2\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.724872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724873 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psgwz\" (UniqueName: \"kubernetes.io/projected/896c42fc-177e-4e20-aa39-efcb2c534269-kube-api-access-psgwz\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724902 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-sys\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724924 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-wtmp\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724952 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-accelerators-collector-config\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.724981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/209a5a3a-a456-41ae-b816-c430bcbb51ce-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.725005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.725034 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/896c42fc-177e-4e20-aa39-efcb2c534269-metrics-client-ca\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.725064 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/209a5a3a-a456-41ae-b816-c430bcbb51ce-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.725090 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.725126 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.725152 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.725228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.725196 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/209a5a3a-a456-41ae-b816-c430bcbb51ce-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.725809 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.725428 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-root\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.726387 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.726358 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.726557 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.726462 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.726668 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.726510 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-sys\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.726781 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.726618 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-wtmp\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.727142 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.727101 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-accelerators-collector-config\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.727291 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.727123 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-textfile\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.728614 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.728097 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/209a5a3a-a456-41ae-b816-c430bcbb51ce-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.728614 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.728511 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/896c42fc-177e-4e20-aa39-efcb2c534269-metrics-client-ca\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.729134 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.729112 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/209a5a3a-a456-41ae-b816-c430bcbb51ce-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.729134 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:54.729129 2582 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 07 18:46:54.729352 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:54.729204 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls podName:4bbfb7b9-88f7-434d-a586-e764e34d5a7b nodeName:}" failed. No retries permitted until 2026-07-07 18:46:55.229184377 +0000 UTC m=+117.254302448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-9nvxd" (UID: "4bbfb7b9-88f7-434d-a586-e764e34d5a7b") : secret "kube-state-metrics-tls" not found Jul 07 18:46:54.729450 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.729410 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.731529 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.731503 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-tls\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.732835 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.732581 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.733483 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.733456 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/209a5a3a-a456-41ae-b816-c430bcbb51ce-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.736905 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.736869 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psgwz\" (UniqueName: \"kubernetes.io/projected/896c42fc-177e-4e20-aa39-efcb2c534269-kube-api-access-psgwz\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.738234 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.738194 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/896c42fc-177e-4e20-aa39-efcb2c534269-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m4ndg\" (UID: \"896c42fc-177e-4e20-aa39-efcb2c534269\") " pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.740545 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.740525 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbjqd\" (UniqueName: \"kubernetes.io/projected/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-api-access-sbjqd\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:54.740626 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.740527 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2dq2\" (UniqueName: \"kubernetes.io/projected/209a5a3a-a456-41ae-b816-c430bcbb51ce-kube-api-access-m2dq2\") pod \"openshift-state-metrics-54cf96dfcf-6zkdt\" (UID: \"209a5a3a-a456-41ae-b816-c430bcbb51ce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.854844 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.854811 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m4ndg" Jul 07 18:46:54.865936 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:54.865904 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" Jul 07 18:46:54.866949 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:54.866924 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896c42fc_177e_4e20_aa39_efcb2c534269.slice/crio-c567d824afbaa406aa2d61f16d662da1ba1d15df89dcef4c8cee569a2fc8f570 WatchSource:0}: Error finding container c567d824afbaa406aa2d61f16d662da1ba1d15df89dcef4c8cee569a2fc8f570: Status 404 returned error can't find the container with id c567d824afbaa406aa2d61f16d662da1ba1d15df89dcef4c8cee569a2fc8f570 Jul 07 18:46:55.017301 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.017270 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt"] Jul 07 18:46:55.019933 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:55.019906 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209a5a3a_a456_41ae_b816_c430bcbb51ce.slice/crio-ee2d1dbe49e49ae640d81018bbd0bdeab009c2115f2407922364466cd0bbc0e6 WatchSource:0}: Error finding container ee2d1dbe49e49ae640d81018bbd0bdeab009c2115f2407922364466cd0bbc0e6: Status 404 returned error can't find the container with id ee2d1dbe49e49ae640d81018bbd0bdeab009c2115f2407922364466cd0bbc0e6 Jul 07 18:46:55.046071 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.046039 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" event={"ID":"209a5a3a-a456-41ae-b816-c430bcbb51ce","Type":"ContainerStarted","Data":"ee2d1dbe49e49ae640d81018bbd0bdeab009c2115f2407922364466cd0bbc0e6"} Jul 07 18:46:55.047167 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.047143 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4ndg" event={"ID":"896c42fc-177e-4e20-aa39-efcb2c534269","Type":"ContainerStarted","Data":"c567d824afbaa406aa2d61f16d662da1ba1d15df89dcef4c8cee569a2fc8f570"} Jul 07 18:46:55.229354 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.229317 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:55.229494 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:55.229474 2582 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 07 18:46:55.229546 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:55.229541 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls podName:4bbfb7b9-88f7-434d-a586-e764e34d5a7b nodeName:}" failed. No retries permitted until 2026-07-07 18:46:56.229526443 +0000 UTC m=+118.254644495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-9nvxd" (UID: "4bbfb7b9-88f7-434d-a586-e764e34d5a7b") : secret "kube-state-metrics-tls" not found Jul 07 18:46:55.614456 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.614426 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:46:55.617999 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.617978 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.620783 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.620758 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 18:46:55.620920 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.620816 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 18:46:55.620920 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.620826 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q2lp2\"" Jul 07 18:46:55.621034 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.620767 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 18:46:55.621034 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.620952 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 18:46:55.621034 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.620995 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 18:46:55.621034 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.620766 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 18:46:55.621227 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.621082 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 18:46:55.621227 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.621085 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 18:46:55.621410 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.621392 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 18:46:55.632807 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.632783 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734330 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734373 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-web-config\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734404 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734457 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734510 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734599 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734628 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtn57\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-kube-api-access-wtn57\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734681 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734714 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-out\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734740 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.740329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.734770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836133 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836096 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836133 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836134 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-web-config\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836158 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836174 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836191 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836343 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836363 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836689 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtn57\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-kube-api-access-wtn57\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836689 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836463 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836689 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-out\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836689 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.836689 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.836582 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.837832 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.837782 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.838977 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.838596 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.841843 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.841483 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-out\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.841843 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.841822 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.842459 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.842297 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.842459 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.842419 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-web-config\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.843328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.843287 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.844460 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.844128 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.844460 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.844416 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.845214 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.845180 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.846082 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.846061 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.846320 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.846299 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.848305 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.848282 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtn57\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-kube-api-access-wtn57\") pod \"alertmanager-main-0\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:55.929185 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:55.929164 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:46:56.009739 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.009695 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-snt25" Jul 07 18:46:56.054432 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.054401 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" event={"ID":"209a5a3a-a456-41ae-b816-c430bcbb51ce","Type":"ContainerStarted","Data":"5d48184635ea3d125c1c37ef58a70da9630ecd30e6e176f61c24aa8297963576"} Jul 07 18:46:56.054621 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.054465 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" event={"ID":"209a5a3a-a456-41ae-b816-c430bcbb51ce","Type":"ContainerStarted","Data":"28b2d70ef2e475fe184a0fb442949666e8402746a5761e9d2b661c3290ff6c26"} Jul 07 18:46:56.056379 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.056328 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4ndg" event={"ID":"896c42fc-177e-4e20-aa39-efcb2c534269","Type":"ContainerStarted","Data":"334db3605427cc2c11f3f3e3f8011ee3f5c12f5e7385fcf1e557ba9b393a7874"} Jul 07 18:46:56.239815 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.239784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:56.242284 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.242258 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bbfb7b9-88f7-434d-a586-e764e34d5a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9nvxd\" (UID: \"4bbfb7b9-88f7-434d-a586-e764e34d5a7b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:56.310261 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.310141 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:46:56.314705 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:56.314678 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93c8acb_2161_4317_9c06_af57bc9ec0e4.slice/crio-58423d41d6341b1a7363f10ad873287e9635b09624daa882a2c01296050307e8 WatchSource:0}: Error finding container 58423d41d6341b1a7363f10ad873287e9635b09624daa882a2c01296050307e8: Status 404 returned error can't find the container with id 58423d41d6341b1a7363f10ad873287e9635b09624daa882a2c01296050307e8 Jul 07 18:46:56.372009 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.371982 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" Jul 07 18:46:56.510169 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:56.510134 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd"] Jul 07 18:46:56.515968 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:56.515934 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbfb7b9_88f7_434d_a586_e764e34d5a7b.slice/crio-df750b414c51b7ea1b56c6bb0a014292256f838c5d570365bed2008a38349b34 WatchSource:0}: Error finding container df750b414c51b7ea1b56c6bb0a014292256f838c5d570365bed2008a38349b34: Status 404 returned error can't find the container with id df750b414c51b7ea1b56c6bb0a014292256f838c5d570365bed2008a38349b34 Jul 07 18:46:57.062373 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.062321 2582 generic.go:356] "Generic (PLEG): container finished" podID="896c42fc-177e-4e20-aa39-efcb2c534269" containerID="334db3605427cc2c11f3f3e3f8011ee3f5c12f5e7385fcf1e557ba9b393a7874" exitCode=0 Jul 07 18:46:57.062856 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.062400 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4ndg" event={"ID":"896c42fc-177e-4e20-aa39-efcb2c534269","Type":"ContainerDied","Data":"334db3605427cc2c11f3f3e3f8011ee3f5c12f5e7385fcf1e557ba9b393a7874"} Jul 07 18:46:57.064443 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.064413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerStarted","Data":"58423d41d6341b1a7363f10ad873287e9635b09624daa882a2c01296050307e8"} Jul 07 18:46:57.066201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.066150 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" event={"ID":"4bbfb7b9-88f7-434d-a586-e764e34d5a7b","Type":"ContainerStarted","Data":"df750b414c51b7ea1b56c6bb0a014292256f838c5d570365bed2008a38349b34"} Jul 07 18:46:57.068887 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.068841 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" event={"ID":"209a5a3a-a456-41ae-b816-c430bcbb51ce","Type":"ContainerStarted","Data":"1911cd27f568dac48a87f3e9322f88ce076a77c81d2a3444d01ebeead3855130"} Jul 07 18:46:57.101663 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.101600 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6zkdt" podStartSLOduration=2.056151767 podStartE2EDuration="3.101583695s" totalImagesPullingTime="1.045431928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:57.100722099 +0000 UTC m=+119.125840185" watchObservedRunningTime="2026-07-07 18:46:57.101583695 +0000 UTC m=+119.126701768" Jul 07 18:46:57.505149 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.505065 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6db7c474f4-cdsjf"] Jul 07 18:46:57.508632 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.508607 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.511331 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.511059 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 18:46:57.511331 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.511075 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9tsfrt87hob3d\"" Jul 07 18:46:57.511331 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.511075 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 18:46:57.511331 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.511166 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 18:46:57.512743 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.512551 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ll2k7\"" Jul 07 18:46:57.512743 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.512565 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 18:46:57.512743 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.512571 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 18:46:57.521486 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.521464 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6db7c474f4-cdsjf"] Jul 07 18:46:57.579493 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.579454 2582 scope.go:117] "RemoveContainer" containerID="a440973c07d5e9feadd5dfc73c9e5b0a92ae959de6534cbac07a59aaa1fd6705" Jul 07 18:46:57.651987 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.651950 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.652160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.651995 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80771db7-4da9-494a-9f3b-0233ef6003c2-metrics-client-ca\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.652160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.652044 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-tls\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.652160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.652093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.652160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.652139 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-grpc-tls\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.652325 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.652171 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h87ll\" (UniqueName: \"kubernetes.io/projected/80771db7-4da9-494a-9f3b-0233ef6003c2-kube-api-access-h87ll\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.652325 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.652244 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.652325 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.652286 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.753018 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.752986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.753148 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.753030 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-grpc-tls\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.753148 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.753061 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h87ll\" (UniqueName: \"kubernetes.io/projected/80771db7-4da9-494a-9f3b-0233ef6003c2-kube-api-access-h87ll\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.753350 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.753324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.754160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.753408 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.754160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.753473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.754160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.753510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80771db7-4da9-494a-9f3b-0233ef6003c2-metrics-client-ca\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.754160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.753564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-tls\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.754565 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.754537 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80771db7-4da9-494a-9f3b-0233ef6003c2-metrics-client-ca\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.756307 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.756233 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-grpc-tls\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.756826 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.756806 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.756826 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.756817 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-tls\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.757021 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.756842 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.757021 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.756964 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.757471 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.757445 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/80771db7-4da9-494a-9f3b-0233ef6003c2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.762453 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.762430 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h87ll\" (UniqueName: \"kubernetes.io/projected/80771db7-4da9-494a-9f3b-0233ef6003c2-kube-api-access-h87ll\") pod \"thanos-querier-6db7c474f4-cdsjf\" (UID: \"80771db7-4da9-494a-9f3b-0233ef6003c2\") " pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:57.819524 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:57.819498 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:46:58.008128 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.007878 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6db7c474f4-cdsjf"] Jul 07 18:46:58.013216 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:58.013175 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80771db7_4da9_494a_9f3b_0233ef6003c2.slice/crio-e863b21f9413572999f076278c654191f1da23656dc88dcaf8c4840a98e7df62 WatchSource:0}: Error finding container e863b21f9413572999f076278c654191f1da23656dc88dcaf8c4840a98e7df62: Status 404 returned error can't find the container with id e863b21f9413572999f076278c654191f1da23656dc88dcaf8c4840a98e7df62 Jul 07 18:46:58.077333 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.076449 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4ndg" event={"ID":"896c42fc-177e-4e20-aa39-efcb2c534269","Type":"ContainerStarted","Data":"3e7ca767195e6826ec03a8a5a3dec7ba550a39a4576ce6bbc8ebfe1f093fdf4f"} Jul 07 18:46:58.077333 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.076511 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4ndg" event={"ID":"896c42fc-177e-4e20-aa39-efcb2c534269","Type":"ContainerStarted","Data":"dadba566b0726b643d9941e30532fa5b45574817c9b97255769b402c0357b09b"} Jul 07 18:46:58.082069 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.080132 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:46:58.082069 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.080215 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" event={"ID":"8383d520-8b64-45f6-b79d-a50b06f0e20c","Type":"ContainerStarted","Data":"9ded377eec7d966154370d2e1afb32295117797e6242083c44123f7e8184b9ea"} Jul 07 18:46:58.090201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.082819 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:58.090201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.085406 2582 generic.go:356] "Generic (PLEG): container finished" podID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerID="8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953" exitCode=0 Jul 07 18:46:58.090201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.085498 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerDied","Data":"8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953"} Jul 07 18:46:58.090201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.087411 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" event={"ID":"80771db7-4da9-494a-9f3b-0233ef6003c2","Type":"ContainerStarted","Data":"e863b21f9413572999f076278c654191f1da23656dc88dcaf8c4840a98e7df62"} Jul 07 18:46:58.090201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.090114 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" event={"ID":"4bbfb7b9-88f7-434d-a586-e764e34d5a7b","Type":"ContainerStarted","Data":"70bb04f46015256ee37ed8f6e0b0571fbed1ccf1ff1e74f760651bda611b43c1"} Jul 07 18:46:58.090201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.090183 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" event={"ID":"4bbfb7b9-88f7-434d-a586-e764e34d5a7b","Type":"ContainerStarted","Data":"429b1cddcf39ddb7477bda9874676c7de096110bb08657ec68ea2e1ff17fbd61"} Jul 07 18:46:58.109321 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.109265 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m4ndg" podStartSLOduration=2.989841445 podStartE2EDuration="4.109245983s" totalImagesPullingTime="1.034970008s" totalInitContainerRuntime="84.43453ms" isStatefulPod=true podCreationTimestamp="2026-07-07 18:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:58.108732738 +0000 UTC m=+120.133850812" watchObservedRunningTime="2026-07-07 18:46:58.109245983 +0000 UTC m=+120.134364058" Jul 07 18:46:58.157659 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.157592 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" podStartSLOduration=40.765757058 podStartE2EDuration="44.157577024s" totalImagesPullingTime="3.391819966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:58.156675784 +0000 UTC m=+120.181793857" watchObservedRunningTime="2026-07-07 18:46:58.157577024 +0000 UTC m=+120.182695096" Jul 07 18:46:58.435015 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.434977 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-wwpsl" Jul 07 18:46:58.625800 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.625760 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-p9njm"] Jul 07 18:46:58.628175 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.628153 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-p9njm" Jul 07 18:46:58.630528 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.630509 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 18:46:58.630528 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.630523 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 18:46:58.630708 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.630553 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-g2vct\"" Jul 07 18:46:58.638894 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.638873 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-p9njm"] Jul 07 18:46:58.763040 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.762964 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjvwq\" (UniqueName: \"kubernetes.io/projected/2396709f-c39c-4aa3-9900-80758e35764e-kube-api-access-vjvwq\") pod \"downloads-685bfb5f64-p9njm\" (UID: \"2396709f-c39c-4aa3-9900-80758e35764e\") " pod="openshift-console/downloads-685bfb5f64-p9njm" Jul 07 18:46:58.863746 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.863709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjvwq\" (UniqueName: \"kubernetes.io/projected/2396709f-c39c-4aa3-9900-80758e35764e-kube-api-access-vjvwq\") pod \"downloads-685bfb5f64-p9njm\" (UID: \"2396709f-c39c-4aa3-9900-80758e35764e\") " pod="openshift-console/downloads-685bfb5f64-p9njm" Jul 07 18:46:58.872799 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.872773 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjvwq\" (UniqueName: \"kubernetes.io/projected/2396709f-c39c-4aa3-9900-80758e35764e-kube-api-access-vjvwq\") pod \"downloads-685bfb5f64-p9njm\" (UID: \"2396709f-c39c-4aa3-9900-80758e35764e\") " pod="openshift-console/downloads-685bfb5f64-p9njm" Jul 07 18:46:58.940239 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:58.940206 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-p9njm" Jul 07 18:46:59.097463 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.097418 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" event={"ID":"4bbfb7b9-88f7-434d-a586-e764e34d5a7b","Type":"ContainerStarted","Data":"4912a507d33d791ad4dbea68dc559b6acea5185c5ce4131191dfe4a1bf6a96ce"} Jul 07 18:46:59.100909 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.100885 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-p9njm"] Jul 07 18:46:59.103871 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:46:59.103847 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2396709f_c39c_4aa3_9900_80758e35764e.slice/crio-252470df80fad47f88029b6dddf1a48ca56f9b3e223b84a2a2e019171aee29f6 WatchSource:0}: Error finding container 252470df80fad47f88029b6dddf1a48ca56f9b3e223b84a2a2e019171aee29f6: Status 404 returned error can't find the container with id 252470df80fad47f88029b6dddf1a48ca56f9b3e223b84a2a2e019171aee29f6 Jul 07 18:46:59.122173 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.122127 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9nvxd" podStartSLOduration=3.786933176 podStartE2EDuration="5.122106584s" totalImagesPullingTime="1.335173408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:46:59.121671054 +0000 UTC m=+121.146789128" watchObservedRunningTime="2026-07-07 18:46:59.122106584 +0000 UTC m=+121.147224660" Jul 07 18:46:59.274274 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.274242 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r"] Jul 07 18:46:59.277838 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.277816 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:46:59.282093 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.282068 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 18:46:59.282215 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.282095 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-k4bsv\"" Jul 07 18:46:59.287472 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.287446 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r"] Jul 07 18:46:59.371350 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.371315 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab2c05e2-daf2-4807-863a-410c44a93324-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bjt8r\" (UID: \"ab2c05e2-daf2-4807-863a-410c44a93324\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:46:59.371522 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.371398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkgvs\" (UniqueName: \"kubernetes.io/projected/ab2c05e2-daf2-4807-863a-410c44a93324-kube-api-access-kkgvs\") pod \"monitoring-plugin-d5dcff7cf-bjt8r\" (UID: \"ab2c05e2-daf2-4807-863a-410c44a93324\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:46:59.472870 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.472826 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab2c05e2-daf2-4807-863a-410c44a93324-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bjt8r\" (UID: \"ab2c05e2-daf2-4807-863a-410c44a93324\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:46:59.473038 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.472898 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkgvs\" (UniqueName: \"kubernetes.io/projected/ab2c05e2-daf2-4807-863a-410c44a93324-kube-api-access-kkgvs\") pod \"monitoring-plugin-d5dcff7cf-bjt8r\" (UID: \"ab2c05e2-daf2-4807-863a-410c44a93324\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:46:59.473038 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:59.473014 2582 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 18:46:59.473142 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:46:59.473094 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2c05e2-daf2-4807-863a-410c44a93324-monitoring-plugin-cert podName:ab2c05e2-daf2-4807-863a-410c44a93324 nodeName:}" failed. No retries permitted until 2026-07-07 18:46:59.973072969 +0000 UTC m=+121.998191022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/ab2c05e2-daf2-4807-863a-410c44a93324-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-bjt8r" (UID: "ab2c05e2-daf2-4807-863a-410c44a93324") : secret "monitoring-plugin-cert" not found Jul 07 18:46:59.485699 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.485669 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkgvs\" (UniqueName: \"kubernetes.io/projected/ab2c05e2-daf2-4807-863a-410c44a93324-kube-api-access-kkgvs\") pod \"monitoring-plugin-d5dcff7cf-bjt8r\" (UID: \"ab2c05e2-daf2-4807-863a-410c44a93324\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:46:59.751303 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.749170 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt"] Jul 07 18:46:59.752541 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.752515 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.759797 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.755810 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 18:46:59.759797 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.756044 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 18:46:59.759797 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.756631 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 18:46:59.759797 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.756876 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8wbsq\"" Jul 07 18:46:59.759797 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.757083 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 18:46:59.759797 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.757269 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 18:46:59.762563 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.762538 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 18:46:59.770703 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.770677 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt"] Jul 07 18:46:59.876982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.876944 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-telemeter-client-tls\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.877153 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.876997 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-serving-certs-ca-bundle\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.877153 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.877093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-metrics-client-ca\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.877292 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.877227 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-secret-telemeter-client\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.877406 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.877286 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.877406 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.877328 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.877406 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.877374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ctsq\" (UniqueName: \"kubernetes.io/projected/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-kube-api-access-7ctsq\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.877560 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.877410 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-federate-client-tls\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.978637 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.978606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab2c05e2-daf2-4807-863a-410c44a93324-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bjt8r\" (UID: \"ab2c05e2-daf2-4807-863a-410c44a93324\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:46:59.978794 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.978640 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-telemeter-client-tls\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.978794 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.978684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-serving-certs-ca-bundle\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.978794 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.978715 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-metrics-client-ca\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.978794 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.978768 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-secret-telemeter-client\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.978995 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.978844 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.978995 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.978869 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.979104 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.978998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ctsq\" (UniqueName: \"kubernetes.io/projected/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-kube-api-access-7ctsq\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.979104 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.979046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-federate-client-tls\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.979958 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.979591 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-metrics-client-ca\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.979958 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.979700 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-serving-certs-ca-bundle\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.980104 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.979985 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.982024 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.981675 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab2c05e2-daf2-4807-863a-410c44a93324-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bjt8r\" (UID: \"ab2c05e2-daf2-4807-863a-410c44a93324\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:46:59.982024 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.981903 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-telemeter-client-tls\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.982024 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.981942 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.982249 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.982083 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-secret-telemeter-client\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.982249 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.982088 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-federate-client-tls\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:46:59.991213 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:46:59.991185 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ctsq\" (UniqueName: \"kubernetes.io/projected/9c7240e5-fcd1-4cd8-9a4f-16c6f476b624-kube-api-access-7ctsq\") pod \"telemeter-client-7c6c54cfcc-dwlvt\" (UID: \"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624\") " pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:47:00.074760 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.074726 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" Jul 07 18:47:00.104640 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.104516 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-p9njm" event={"ID":"2396709f-c39c-4aa3-9900-80758e35764e","Type":"ContainerStarted","Data":"252470df80fad47f88029b6dddf1a48ca56f9b3e223b84a2a2e019171aee29f6"} Jul 07 18:47:00.191662 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.191603 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:47:00.589122 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.589090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r"] Jul 07 18:47:00.601179 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:47:00.601156 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab2c05e2_daf2_4807_863a_410c44a93324.slice/crio-097e67ec36d3b0a20674610bb2c72dd5589224ba4f4e7438e498c4069773c705 WatchSource:0}: Error finding container 097e67ec36d3b0a20674610bb2c72dd5589224ba4f4e7438e498c4069773c705: Status 404 returned error can't find the container with id 097e67ec36d3b0a20674610bb2c72dd5589224ba4f4e7438e498c4069773c705 Jul 07 18:47:00.617382 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.617350 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt"] Jul 07 18:47:00.635159 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:47:00.634365 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7240e5_fcd1_4cd8_9a4f_16c6f476b624.slice/crio-ac3ae6e03ca10262feec327d9a00cd93552c07cca590d5eff8efd7306720669e WatchSource:0}: Error finding container ac3ae6e03ca10262feec327d9a00cd93552c07cca590d5eff8efd7306720669e: Status 404 returned error can't find the container with id ac3ae6e03ca10262feec327d9a00cd93552c07cca590d5eff8efd7306720669e Jul 07 18:47:00.851843 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.851813 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:47:00.855268 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.855249 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.857982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.857789 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 18:47:00.857982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.857789 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 18:47:00.857982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.857855 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-s7gandu7pon0\"" Jul 07 18:47:00.858198 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.858170 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 18:47:00.858250 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.858222 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 18:47:00.858428 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.858416 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 18:47:00.858494 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.858418 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-c2g8k\"" Jul 07 18:47:00.858620 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.858603 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 18:47:00.858702 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.858690 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 18:47:00.858932 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.858911 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 18:47:00.858932 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.858923 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 18:47:00.859071 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.859059 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 18:47:00.859528 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.859179 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 18:47:00.859528 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.859351 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 18:47:00.859528 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.859429 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 18:47:00.862039 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.862013 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 18:47:00.864734 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.864716 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 18:47:00.881166 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.881139 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:47:00.990440 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990361 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990440 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990396 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990440 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990432 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-web-config\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990726 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990522 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990726 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990558 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990726 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990586 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-config\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990726 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990609 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990726 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990726 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990681 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990737 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990778 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbqpk\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-kube-api-access-zbqpk\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990808 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990837 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.990982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990960 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.991201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.990984 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.991201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.991006 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-config-out\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.991201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.991033 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:00.991201 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:00.991057 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.091972 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-config-out\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092081 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092120 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092200 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-web-config\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092288 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092315 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092345 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-config\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092393 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092425 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbqpk\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-kube-api-access-zbqpk\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.093982 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092537 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.094891 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.094891 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092586 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.094891 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.092615 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.094891 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.093288 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.097240 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.097055 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.099251 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.098341 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.099251 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.098941 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.103691 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.101121 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.103691 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.101967 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.103691 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.102232 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.105414 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.105255 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.107218 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.107151 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.108111 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.108005 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.108111 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.108069 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-config\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.108802 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.108755 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-web-config\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.109393 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.109346 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.111349 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.110178 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.111349 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.110535 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.111349 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.110561 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.112800 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.112777 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.114737 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.114592 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-config-out\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.115015 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.114992 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbqpk\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-kube-api-access-zbqpk\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.115450 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.115381 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.119288 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.119260 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerStarted","Data":"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1"} Jul 07 18:47:01.119386 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.119343 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerStarted","Data":"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb"} Jul 07 18:47:01.119386 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.119374 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerStarted","Data":"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953"} Jul 07 18:47:01.119496 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.119413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerStarted","Data":"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab"} Jul 07 18:47:01.119496 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.119454 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerStarted","Data":"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1"} Jul 07 18:47:01.122594 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.122397 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" event={"ID":"80771db7-4da9-494a-9f3b-0233ef6003c2","Type":"ContainerStarted","Data":"53a7d77861d9cdadbf3a37a3b609495c3d6751a49f9e720687bd26f6f9e351ca"} Jul 07 18:47:01.122594 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.122465 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" event={"ID":"80771db7-4da9-494a-9f3b-0233ef6003c2","Type":"ContainerStarted","Data":"26f91988eff32d631e697b5b3c873621387c78936964b9d42ccdcd3cd0ee9ced"} Jul 07 18:47:01.122594 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.122509 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" event={"ID":"80771db7-4da9-494a-9f3b-0233ef6003c2","Type":"ContainerStarted","Data":"9a6fc0c9c7754919d23301e41a44c4bbbd8bc00846956c06a44c796b52594ed8"} Jul 07 18:47:01.125952 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.125928 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" event={"ID":"ab2c05e2-daf2-4807-863a-410c44a93324","Type":"ContainerStarted","Data":"097e67ec36d3b0a20674610bb2c72dd5589224ba4f4e7438e498c4069773c705"} Jul 07 18:47:01.128521 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.128456 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" event={"ID":"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624","Type":"ContainerStarted","Data":"ac3ae6e03ca10262feec327d9a00cd93552c07cca590d5eff8efd7306720669e"} Jul 07 18:47:01.167375 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.167317 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:01.365568 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:01.365462 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:47:01.370307 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:47:01.370269 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode34479f2_1632_4876_bdb9_bda323223879.slice/crio-65fc9b1f06add8ca6271fa82883c320e7dd299e41b94d8d8cf42b0f992daafef WatchSource:0}: Error finding container 65fc9b1f06add8ca6271fa82883c320e7dd299e41b94d8d8cf42b0f992daafef: Status 404 returned error can't find the container with id 65fc9b1f06add8ca6271fa82883c320e7dd299e41b94d8d8cf42b0f992daafef Jul 07 18:47:02.138373 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:02.137005 2582 generic.go:356] "Generic (PLEG): container finished" podID="e34479f2-1632-4876-bdb9-bda323223879" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" exitCode=0 Jul 07 18:47:02.138373 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:02.137123 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerDied","Data":"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc"} Jul 07 18:47:02.138373 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:02.137231 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerStarted","Data":"65fc9b1f06add8ca6271fa82883c320e7dd299e41b94d8d8cf42b0f992daafef"} Jul 07 18:47:02.149684 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:02.148396 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerStarted","Data":"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8"} Jul 07 18:47:02.153056 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:02.153024 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" event={"ID":"80771db7-4da9-494a-9f3b-0233ef6003c2","Type":"ContainerStarted","Data":"775d6ea839ae614e3945d0e6d021a65f5945b37065260e626ea705935828a87f"} Jul 07 18:47:02.197723 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:02.197459 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.466582938 podStartE2EDuration="7.197435242s" totalImagesPullingTime="4.702899345s" totalInitContainerRuntime="27.952959ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:02.196539143 +0000 UTC m=+124.221657230" watchObservedRunningTime="2026-07-07 18:47:02.197435242 +0000 UTC m=+124.222553317" Jul 07 18:47:03.162203 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.162163 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" event={"ID":"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624","Type":"ContainerStarted","Data":"5bc1890cdcdc33f28b8d463951dc51c9fb16e6177d82be189a5ac97f7c4e2e55"} Jul 07 18:47:03.162616 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.162233 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" event={"ID":"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624","Type":"ContainerStarted","Data":"fdc4e42e805a2e77a016f1c44148388f4cf83a73b0e40c249a23fb6f9fd6314c"} Jul 07 18:47:03.162616 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.162260 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" event={"ID":"9c7240e5-fcd1-4cd8-9a4f-16c6f476b624","Type":"ContainerStarted","Data":"79c51ae075d72e6eb694e7b7468bf7c0e1e6d553237c7372791057059e6f1912"} Jul 07 18:47:03.166480 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.166448 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" event={"ID":"80771db7-4da9-494a-9f3b-0233ef6003c2","Type":"ContainerStarted","Data":"eebcff1a938bf3c14fb84391d1b03a072e640dacdded1254512c2b8bf10098c6"} Jul 07 18:47:03.166627 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.166507 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" event={"ID":"80771db7-4da9-494a-9f3b-0233ef6003c2","Type":"ContainerStarted","Data":"2f3ad823a463f0067a57c8f5b477efae1de58caa457977e5e0f68602f8390503"} Jul 07 18:47:03.166627 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.166596 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:47:03.169860 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.169833 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" event={"ID":"ab2c05e2-daf2-4807-863a-410c44a93324","Type":"ContainerStarted","Data":"2fb2de35c1c5ce30fd57e7642651a56e1bc2706c217c865155698ccf1077cf85"} Jul 07 18:47:03.170463 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.170443 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:47:03.177675 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.177228 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" Jul 07 18:47:03.197333 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.197275 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c6c54cfcc-dwlvt" podStartSLOduration=2.027147831 podStartE2EDuration="4.197254789s" totalImagesPullingTime="2.170106958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:03.196220601 +0000 UTC m=+125.221338674" watchObservedRunningTime="2026-07-07 18:47:03.197254789 +0000 UTC m=+125.222372866" Jul 07 18:47:03.229111 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.229054 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" podStartSLOduration=2.8362461420000002 podStartE2EDuration="6.229035857s" totalImagesPullingTime="3.392789715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:03.227233083 +0000 UTC m=+125.252351212" watchObservedRunningTime="2026-07-07 18:47:03.229035857 +0000 UTC m=+125.254153931" Jul 07 18:47:03.246283 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:03.245974 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bjt8r" podStartSLOduration=2.068926779 podStartE2EDuration="4.245957886s" totalImagesPullingTime="2.177031107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:03.245437259 +0000 UTC m=+125.270555332" watchObservedRunningTime="2026-07-07 18:47:03.245957886 +0000 UTC m=+125.271076153" Jul 07 18:47:06.196539 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:06.196498 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerStarted","Data":"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e"} Jul 07 18:47:06.196972 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:06.196594 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerStarted","Data":"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8"} Jul 07 18:47:06.196972 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:06.196634 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerStarted","Data":"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e"} Jul 07 18:47:07.206264 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:07.206226 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerStarted","Data":"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c"} Jul 07 18:47:07.206692 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:07.206295 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerStarted","Data":"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558"} Jul 07 18:47:07.206692 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:07.206323 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerStarted","Data":"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e"} Jul 07 18:47:07.241065 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:07.241000 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.466108286 podStartE2EDuration="7.240981641s" totalImagesPullingTime="3.749455652s" totalInitContainerRuntime="25.417703ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:47:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:07.239705707 +0000 UTC m=+129.264823805" watchObservedRunningTime="2026-07-07 18:47:07.240981641 +0000 UTC m=+129.266099714" Jul 07 18:47:08.169090 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:08.169048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:47:08.171546 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:08.171513 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19e9d15b-2047-4bb5-846d-dd0ffe63d8cd-metrics-certs\") pod \"network-metrics-daemon-srz9n\" (UID: \"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd\") " pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:47:08.315829 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:08.315778 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8gxh7\"" Jul 07 18:47:08.321155 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:08.321130 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srz9n" Jul 07 18:47:08.479312 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:08.479290 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-srz9n"] Jul 07 18:47:08.481953 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:47:08.481923 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e9d15b_2047_4bb5_846d_dd0ffe63d8cd.slice/crio-b8c02ee5e61f7ab2ff09123567a28a1457c381b166607a834bf0b7a202a39567 WatchSource:0}: Error finding container b8c02ee5e61f7ab2ff09123567a28a1457c381b166607a834bf0b7a202a39567: Status 404 returned error can't find the container with id b8c02ee5e61f7ab2ff09123567a28a1457c381b166607a834bf0b7a202a39567 Jul 07 18:47:09.185517 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:09.185486 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6db7c474f4-cdsjf" Jul 07 18:47:09.218444 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:09.218396 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-srz9n" event={"ID":"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd","Type":"ContainerStarted","Data":"b8c02ee5e61f7ab2ff09123567a28a1457c381b166607a834bf0b7a202a39567"} Jul 07 18:47:10.225411 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:10.225365 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-srz9n" event={"ID":"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd","Type":"ContainerStarted","Data":"6639fc177c33cbf60ecc4737729400c1037ed3c2c9937163554b676d3c6545ce"} Jul 07 18:47:10.225838 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:10.225431 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-srz9n" event={"ID":"19e9d15b-2047-4bb5-846d-dd0ffe63d8cd","Type":"ContainerStarted","Data":"fa355eae7c4c9a63165b7ff10ac2d900a9ab56ebb35573f16dc6a844ddc55f81"} Jul 07 18:47:10.248269 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:10.248218 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-srz9n" podStartSLOduration=131.234318193 podStartE2EDuration="2m12.248199451s" totalImagesPullingTime="1.013881258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:10.245451789 +0000 UTC m=+132.270569863" watchObservedRunningTime="2026-07-07 18:47:10.248199451 +0000 UTC m=+132.273317524" Jul 07 18:47:11.168377 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:11.168342 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:47:19.279354 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:19.279309 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-p9njm" event={"ID":"2396709f-c39c-4aa3-9900-80758e35764e","Type":"ContainerStarted","Data":"2f570094aa0383fb74fea39d20cc695440b7f575eb7dfc8d1601597e29beb666"} Jul 07 18:47:19.279899 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:19.279547 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-p9njm" Jul 07 18:47:19.291809 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:19.291778 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-p9njm" Jul 07 18:47:19.302631 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:19.302565 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-p9njm" podStartSLOduration=1.851490805 podStartE2EDuration="21.302544902s" totalImagesPullingTime="19.451054097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:47:19.301486022 +0000 UTC m=+141.326604128" watchObservedRunningTime="2026-07-07 18:47:19.302544902 +0000 UTC m=+141.327662978" Jul 07 18:47:29.332430 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:29.332392 2582 generic.go:356] "Generic (PLEG): container finished" podID="32ea9797-0ffc-466c-82ec-4a6217043139" containerID="b6af83f2b9f3f4de42fd0f331b734e22e09636bcfafbae895fd0cc87fdd0c606" exitCode=0 Jul 07 18:47:29.332944 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:29.332466 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" event={"ID":"32ea9797-0ffc-466c-82ec-4a6217043139","Type":"ContainerDied","Data":"b6af83f2b9f3f4de42fd0f331b734e22e09636bcfafbae895fd0cc87fdd0c606"} Jul 07 18:47:29.332944 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:29.332834 2582 scope.go:117] "RemoveContainer" containerID="b6af83f2b9f3f4de42fd0f331b734e22e09636bcfafbae895fd0cc87fdd0c606" Jul 07 18:47:30.338569 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:30.338535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-5vggq" event={"ID":"32ea9797-0ffc-466c-82ec-4a6217043139","Type":"ContainerStarted","Data":"9e47b4eded583eba2b6209173c48c09063f8cf2af3229405d3deab68ce2ef332"} Jul 07 18:47:44.399841 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:44.399809 2582 generic.go:356] "Generic (PLEG): container finished" podID="4c436eb0-0acc-4322-9ab0-3611167e3e47" containerID="3eb0e30a4edcdd778feafb8ddd70684e25ab1ee29bc91bbb0a6a1d52aae41c69" exitCode=0 Jul 07 18:47:44.400244 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:44.399847 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" event={"ID":"4c436eb0-0acc-4322-9ab0-3611167e3e47","Type":"ContainerDied","Data":"3eb0e30a4edcdd778feafb8ddd70684e25ab1ee29bc91bbb0a6a1d52aae41c69"} Jul 07 18:47:44.400244 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:44.400139 2582 scope.go:117] "RemoveContainer" containerID="3eb0e30a4edcdd778feafb8ddd70684e25ab1ee29bc91bbb0a6a1d52aae41c69" Jul 07 18:47:45.405968 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:47:45.405934 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7tln" event={"ID":"4c436eb0-0acc-4322-9ab0-3611167e3e47","Type":"ContainerStarted","Data":"186c82e55d273adedaa289b3fc4119279d7825f9f5db9ae5ca0a479bbd01892d"} Jul 07 18:48:01.167879 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:01.167788 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:01.233592 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:01.233565 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:01.502902 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:01.502826 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:14.990700 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:14.990660 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:48:14.991143 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:14.990973 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="alertmanager" containerID="cri-o://825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1" gracePeriod=120 Jul 07 18:48:14.991143 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:14.991013 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy-web" containerID="cri-o://b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953" gracePeriod=120 Jul 07 18:48:14.991143 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:14.991030 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="prom-label-proxy" containerID="cri-o://d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8" gracePeriod=120 Jul 07 18:48:14.991143 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:14.991024 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="config-reloader" containerID="cri-o://21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab" gracePeriod=120 Jul 07 18:48:14.991143 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:14.990988 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy" containerID="cri-o://c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb" gracePeriod=120 Jul 07 18:48:14.991389 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:14.991234 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy-metric" containerID="cri-o://b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1" gracePeriod=120 Jul 07 18:48:15.548295 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:15.548262 2582 generic.go:356] "Generic (PLEG): container finished" podID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerID="d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8" exitCode=0 Jul 07 18:48:15.548295 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:15.548286 2582 generic.go:356] "Generic (PLEG): container finished" podID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerID="c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb" exitCode=0 Jul 07 18:48:15.548295 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:15.548293 2582 generic.go:356] "Generic (PLEG): container finished" podID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerID="21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab" exitCode=0 Jul 07 18:48:15.548295 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:15.548299 2582 generic.go:356] "Generic (PLEG): container finished" podID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerID="825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1" exitCode=0 Jul 07 18:48:15.548564 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:15.548328 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerDied","Data":"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8"} Jul 07 18:48:15.548564 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:15.548388 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerDied","Data":"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb"} Jul 07 18:48:15.548564 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:15.548415 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerDied","Data":"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab"} Jul 07 18:48:15.548564 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:15.548440 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerDied","Data":"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1"} Jul 07 18:48:16.238610 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.238587 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:48:16.393723 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.393695 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-main-tls\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.393898 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.393741 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-trusted-ca-bundle\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.393898 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.393765 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-out\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.393990 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.393955 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-main-db\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394045 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394012 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394045 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394040 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-tls-assets\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394214 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394061 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-web-config\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394214 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394079 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-cluster-tls-config\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394214 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394101 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-metrics-client-ca\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394214 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394164 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtn57\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-kube-api-access-wtn57\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394214 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394200 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-volume\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394471 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394149 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:16.394471 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394234 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.394471 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394280 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:48:16.394471 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394298 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy\") pod \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\" (UID: \"c93c8acb-2161-4317-9c06-af57bc9ec0e4\") " Jul 07 18:48:16.395267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394703 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:16.395267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394820 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.395267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394842 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-alertmanager-main-db\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.395267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.394858 2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c93c8acb-2161-4317-9c06-af57bc9ec0e4-metrics-client-ca\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.396809 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.396780 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:16.396905 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.396868 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:16.396905 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.396880 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-out" (OuterVolumeSpecName: "config-out") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:48:16.398027 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.397983 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:16.399436 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.399397 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:16.399436 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.399420 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-kube-api-access-wtn57" (OuterVolumeSpecName: "kube-api-access-wtn57") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "kube-api-access-wtn57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:48:16.399436 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.399424 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:48:16.399666 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.399594 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-volume" (OuterVolumeSpecName: "config-volume") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:16.404368 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.404344 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:16.411769 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.411747 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-web-config" (OuterVolumeSpecName: "web-config") pod "c93c8acb-2161-4317-9c06-af57bc9ec0e4" (UID: "c93c8acb-2161-4317-9c06-af57bc9ec0e4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:16.495823 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495792 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.495823 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495825 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.496008 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495838 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-main-tls\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.496008 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495854 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-out\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.496008 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495868 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.496008 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495881 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-tls-assets\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.496008 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495893 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-web-config\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.496008 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495906 2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-cluster-tls-config\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.496008 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495917 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtn57\" (UniqueName: \"kubernetes.io/projected/c93c8acb-2161-4317-9c06-af57bc9ec0e4-kube-api-access-wtn57\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.496008 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.495929 2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c93c8acb-2161-4317-9c06-af57bc9ec0e4-config-volume\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:16.555896 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.555859 2582 generic.go:356] "Generic (PLEG): container finished" podID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerID="b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1" exitCode=0 Jul 07 18:48:16.555896 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.555890 2582 generic.go:356] "Generic (PLEG): container finished" podID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerID="b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953" exitCode=0 Jul 07 18:48:16.556091 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.555939 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 18:48:16.556091 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.555934 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerDied","Data":"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1"} Jul 07 18:48:16.556091 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.556080 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerDied","Data":"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953"} Jul 07 18:48:16.556224 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.556115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c93c8acb-2161-4317-9c06-af57bc9ec0e4","Type":"ContainerDied","Data":"58423d41d6341b1a7363f10ad873287e9635b09624daa882a2c01296050307e8"} Jul 07 18:48:16.556224 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.556152 2582 scope.go:117] "RemoveContainer" containerID="d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8" Jul 07 18:48:16.564366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.564346 2582 scope.go:117] "RemoveContainer" containerID="b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1" Jul 07 18:48:16.574065 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.574026 2582 scope.go:117] "RemoveContainer" containerID="c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb" Jul 07 18:48:16.581873 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.581855 2582 scope.go:117] "RemoveContainer" containerID="b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953" Jul 07 18:48:16.587685 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.587662 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:48:16.590924 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.590873 2582 scope.go:117] "RemoveContainer" containerID="21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab" Jul 07 18:48:16.593012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.592983 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 18:48:16.598428 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.598408 2582 scope.go:117] "RemoveContainer" containerID="825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1" Jul 07 18:48:16.605715 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.605696 2582 scope.go:117] "RemoveContainer" containerID="8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953" Jul 07 18:48:16.612813 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.612789 2582 scope.go:117] "RemoveContainer" containerID="d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8" Jul 07 18:48:16.613079 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:16.613058 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8\": container with ID starting with d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8 not found: ID does not exist" containerID="d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8" Jul 07 18:48:16.613130 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.613088 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8"} err="failed to get container status \"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8\": rpc error: code = NotFound desc = could not find container \"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8\": container with ID starting with d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8 not found: ID does not exist" Jul 07 18:48:16.613130 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.613122 2582 scope.go:117] "RemoveContainer" containerID="b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1" Jul 07 18:48:16.613371 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:16.613354 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1\": container with ID starting with b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1 not found: ID does not exist" containerID="b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1" Jul 07 18:48:16.613437 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.613380 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1"} err="failed to get container status \"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1\": rpc error: code = NotFound desc = could not find container \"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1\": container with ID starting with b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1 not found: ID does not exist" Jul 07 18:48:16.613437 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.613403 2582 scope.go:117] "RemoveContainer" containerID="c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb" Jul 07 18:48:16.613697 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:16.613676 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb\": container with ID starting with c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb not found: ID does not exist" containerID="c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb" Jul 07 18:48:16.613789 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.613699 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb"} err="failed to get container status \"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb\": rpc error: code = NotFound desc = could not find container \"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb\": container with ID starting with c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb not found: ID does not exist" Jul 07 18:48:16.613789 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.613714 2582 scope.go:117] "RemoveContainer" containerID="b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953" Jul 07 18:48:16.613917 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:16.613902 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953\": container with ID starting with b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953 not found: ID does not exist" containerID="b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953" Jul 07 18:48:16.613954 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.613922 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953"} err="failed to get container status \"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953\": rpc error: code = NotFound desc = could not find container \"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953\": container with ID starting with b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953 not found: ID does not exist" Jul 07 18:48:16.613954 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.613934 2582 scope.go:117] "RemoveContainer" containerID="21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab" Jul 07 18:48:16.614178 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:16.614161 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab\": container with ID starting with 21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab not found: ID does not exist" containerID="21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab" Jul 07 18:48:16.614228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.614180 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab"} err="failed to get container status \"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab\": rpc error: code = NotFound desc = could not find container \"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab\": container with ID starting with 21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab not found: ID does not exist" Jul 07 18:48:16.614228 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.614192 2582 scope.go:117] "RemoveContainer" containerID="825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1" Jul 07 18:48:16.614411 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:16.614392 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1\": container with ID starting with 825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1 not found: ID does not exist" containerID="825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1" Jul 07 18:48:16.614452 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.614415 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1"} err="failed to get container status \"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1\": rpc error: code = NotFound desc = could not find container \"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1\": container with ID starting with 825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1 not found: ID does not exist" Jul 07 18:48:16.614452 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.614427 2582 scope.go:117] "RemoveContainer" containerID="8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953" Jul 07 18:48:16.614625 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:16.614604 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953\": container with ID starting with 8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953 not found: ID does not exist" containerID="8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953" Jul 07 18:48:16.614703 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.614629 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953"} err="failed to get container status \"8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953\": rpc error: code = NotFound desc = could not find container \"8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953\": container with ID starting with 8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953 not found: ID does not exist" Jul 07 18:48:16.614703 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.614660 2582 scope.go:117] "RemoveContainer" containerID="d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8" Jul 07 18:48:16.614865 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.614848 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8"} err="failed to get container status \"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8\": rpc error: code = NotFound desc = could not find container \"d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8\": container with ID starting with d1f1249e06d9c0ec81a9112b92fe2eb3e729769967c56222ecfd93ee6136d9a8 not found: ID does not exist" Jul 07 18:48:16.614908 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.614865 2582 scope.go:117] "RemoveContainer" containerID="b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1" Jul 07 18:48:16.615057 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615035 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1"} err="failed to get container status \"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1\": rpc error: code = NotFound desc = could not find container \"b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1\": container with ID starting with b9c77b3db1ae551e8dc95ee1e10233708ae8d4d74b1c66739ab10cd3549a6af1 not found: ID does not exist" Jul 07 18:48:16.615131 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615058 2582 scope.go:117] "RemoveContainer" containerID="c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb" Jul 07 18:48:16.615279 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615259 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb"} err="failed to get container status \"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb\": rpc error: code = NotFound desc = could not find container \"c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb\": container with ID starting with c581dbb33a97bbf3d60abf234ac8f8901f39b0f235bf4c69b98f253147f37bbb not found: ID does not exist" Jul 07 18:48:16.615333 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615281 2582 scope.go:117] "RemoveContainer" containerID="b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953" Jul 07 18:48:16.615467 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615448 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953"} err="failed to get container status \"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953\": rpc error: code = NotFound desc = could not find container \"b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953\": container with ID starting with b10e5266330b0861bd0c87a61656eee4daa072b102a1caab7a2e4b25b689b953 not found: ID does not exist" Jul 07 18:48:16.615561 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615469 2582 scope.go:117] "RemoveContainer" containerID="21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab" Jul 07 18:48:16.615682 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615658 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab"} err="failed to get container status \"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab\": rpc error: code = NotFound desc = could not find container \"21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab\": container with ID starting with 21fcec8bfb57a370009c75e8803fd7249e8099c32d95da16379c04cd835f69ab not found: ID does not exist" Jul 07 18:48:16.615747 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615685 2582 scope.go:117] "RemoveContainer" containerID="825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1" Jul 07 18:48:16.615908 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615893 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1"} err="failed to get container status \"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1\": rpc error: code = NotFound desc = could not find container \"825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1\": container with ID starting with 825e7f6639b279bd6738a12097dd02f3a119f7138e7fc2ff5ccbf9f8d22e74b1 not found: ID does not exist" Jul 07 18:48:16.615955 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.615909 2582 scope.go:117] "RemoveContainer" containerID="8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953" Jul 07 18:48:16.616100 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:16.616081 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953"} err="failed to get container status \"8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953\": rpc error: code = NotFound desc = could not find container \"8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953\": container with ID starting with 8bedb1d8eb6e91041856a38d60dd2e4b435767f2aa04ee7b5aec54fb71e82953 not found: ID does not exist" Jul 07 18:48:18.584357 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:18.584327 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" path="/var/lib/kubelet/pods/c93c8acb-2161-4317-9c06-af57bc9ec0e4/volumes" Jul 07 18:48:19.291321 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.291290 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:48:19.291639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.291596 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="prometheus" containerID="cri-o://75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" gracePeriod=600 Jul 07 18:48:19.291639 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.291625 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy-thanos" containerID="cri-o://24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" gracePeriod=600 Jul 07 18:48:19.291889 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.291689 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="config-reloader" containerID="cri-o://8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" gracePeriod=600 Jul 07 18:48:19.291889 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.291673 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="thanos-sidecar" containerID="cri-o://1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" gracePeriod=600 Jul 07 18:48:19.291889 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.291606 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy" containerID="cri-o://06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" gracePeriod=600 Jul 07 18:48:19.291889 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.291751 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy-web" containerID="cri-o://f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" gracePeriod=600 Jul 07 18:48:19.535206 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.535181 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:19.580269 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580240 2582 generic.go:356] "Generic (PLEG): container finished" podID="e34479f2-1632-4876-bdb9-bda323223879" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" exitCode=0 Jul 07 18:48:19.580269 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580265 2582 generic.go:356] "Generic (PLEG): container finished" podID="e34479f2-1632-4876-bdb9-bda323223879" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" exitCode=0 Jul 07 18:48:19.580269 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580271 2582 generic.go:356] "Generic (PLEG): container finished" podID="e34479f2-1632-4876-bdb9-bda323223879" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" exitCode=0 Jul 07 18:48:19.580269 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580277 2582 generic.go:356] "Generic (PLEG): container finished" podID="e34479f2-1632-4876-bdb9-bda323223879" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" exitCode=0 Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580282 2582 generic.go:356] "Generic (PLEG): container finished" podID="e34479f2-1632-4876-bdb9-bda323223879" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" exitCode=0 Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580287 2582 generic.go:356] "Generic (PLEG): container finished" podID="e34479f2-1632-4876-bdb9-bda323223879" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" exitCode=0 Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580306 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerDied","Data":"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c"} Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580329 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580353 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerDied","Data":"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558"} Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580380 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerDied","Data":"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e"} Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580404 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerDied","Data":"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e"} Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580433 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerDied","Data":"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8"} Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580430 2582 scope.go:117] "RemoveContainer" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580458 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerDied","Data":"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e"} Jul 07 18:48:19.580596 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.580481 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e34479f2-1632-4876-bdb9-bda323223879","Type":"ContainerDied","Data":"65fc9b1f06add8ca6271fa82883c320e7dd299e41b94d8d8cf42b0f992daafef"} Jul 07 18:48:19.588917 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.588897 2582 scope.go:117] "RemoveContainer" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" Jul 07 18:48:19.596482 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.596454 2582 scope.go:117] "RemoveContainer" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" Jul 07 18:48:19.605083 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.605068 2582 scope.go:117] "RemoveContainer" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" Jul 07 18:48:19.612052 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.612036 2582 scope.go:117] "RemoveContainer" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" Jul 07 18:48:19.620149 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.620132 2582 scope.go:117] "RemoveContainer" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" Jul 07 18:48:19.623353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623319 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-metrics-client-ca\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.623353 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623351 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-serving-certs-ca-bundle\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.623499 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623372 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-metrics-client-certs\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.623499 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623397 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-tls-assets\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.623725 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623701 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-kubelet-serving-ca-bundle\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.623800 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623743 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:19.623800 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623755 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-1\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.623909 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623824 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:19.623909 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623878 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-config\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624063 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623917 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624063 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.623964 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-thanos-prometheus-http-client-file\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624063 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624001 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-grpc-tls\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624063 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624013 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:19.624063 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624044 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-2\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624080 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-kube-rbac-proxy\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624122 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624132 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-trusted-ca-bundle\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624162 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbqpk\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-kube-api-access-zbqpk\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624193 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-0\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624225 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-db\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624253 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-web-config\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624280 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.624316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624306 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-tls\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.625870 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624333 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-config-out\") pod \"e34479f2-1632-4876-bdb9-bda323223879\" (UID: \"e34479f2-1632-4876-bdb9-bda323223879\") " Jul 07 18:48:19.625870 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624602 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-metrics-client-ca\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.625870 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624622 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.625870 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624636 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.625870 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.624938 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.626395 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.626360 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:19.627166 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.626542 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:48:19.627166 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.626957 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:19.627166 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.627028 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.627665 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.627542 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.627665 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.627562 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:48:19.627804 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.627753 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-config" (OuterVolumeSpecName: "config") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.628158 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.628135 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:48:19.628243 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.628217 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.628300 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.628236 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-config-out" (OuterVolumeSpecName: "config-out") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:48:19.629404 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.629369 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.629504 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.629407 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.629504 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.629432 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-kube-api-access-zbqpk" (OuterVolumeSpecName: "kube-api-access-zbqpk") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "kube-api-access-zbqpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:48:19.630499 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.630481 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.630814 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.630791 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.631079 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.631066 2582 scope.go:117] "RemoveContainer" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" Jul 07 18:48:19.638267 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.638249 2582 scope.go:117] "RemoveContainer" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" Jul 07 18:48:19.638520 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:19.638499 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": container with ID starting with 24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c not found: ID does not exist" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" Jul 07 18:48:19.638560 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.638529 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c"} err="failed to get container status \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": rpc error: code = NotFound desc = could not find container \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": container with ID starting with 24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c not found: ID does not exist" Jul 07 18:48:19.638560 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.638547 2582 scope.go:117] "RemoveContainer" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" Jul 07 18:48:19.638894 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:19.638874 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": container with ID starting with 06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558 not found: ID does not exist" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" Jul 07 18:48:19.638970 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.638895 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558"} err="failed to get container status \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": rpc error: code = NotFound desc = could not find container \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": container with ID starting with 06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558 not found: ID does not exist" Jul 07 18:48:19.638970 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.638909 2582 scope.go:117] "RemoveContainer" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" Jul 07 18:48:19.639174 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:19.639157 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": container with ID starting with f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e not found: ID does not exist" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" Jul 07 18:48:19.639217 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.639180 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e"} err="failed to get container status \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": rpc error: code = NotFound desc = could not find container \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": container with ID starting with f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e not found: ID does not exist" Jul 07 18:48:19.639217 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.639192 2582 scope.go:117] "RemoveContainer" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" Jul 07 18:48:19.639583 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:19.639569 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": container with ID starting with 1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e not found: ID does not exist" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" Jul 07 18:48:19.639631 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.639587 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e"} err="failed to get container status \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": rpc error: code = NotFound desc = could not find container \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": container with ID starting with 1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e not found: ID does not exist" Jul 07 18:48:19.639631 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.639600 2582 scope.go:117] "RemoveContainer" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" Jul 07 18:48:19.639913 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:19.639899 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": container with ID starting with 8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8 not found: ID does not exist" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" Jul 07 18:48:19.639954 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.639917 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8"} err="failed to get container status \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": rpc error: code = NotFound desc = could not find container \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": container with ID starting with 8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8 not found: ID does not exist" Jul 07 18:48:19.639954 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.639928 2582 scope.go:117] "RemoveContainer" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" Jul 07 18:48:19.640119 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:19.640098 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": container with ID starting with 75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e not found: ID does not exist" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" Jul 07 18:48:19.640184 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640128 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e"} err="failed to get container status \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": rpc error: code = NotFound desc = could not find container \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": container with ID starting with 75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e not found: ID does not exist" Jul 07 18:48:19.640184 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640150 2582 scope.go:117] "RemoveContainer" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" Jul 07 18:48:19.640374 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:48:19.640360 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": container with ID starting with b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc not found: ID does not exist" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" Jul 07 18:48:19.640411 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640377 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc"} err="failed to get container status \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": rpc error: code = NotFound desc = could not find container \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": container with ID starting with b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc not found: ID does not exist" Jul 07 18:48:19.640411 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640389 2582 scope.go:117] "RemoveContainer" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" Jul 07 18:48:19.640615 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640599 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c"} err="failed to get container status \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": rpc error: code = NotFound desc = could not find container \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": container with ID starting with 24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c not found: ID does not exist" Jul 07 18:48:19.640709 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640617 2582 scope.go:117] "RemoveContainer" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" Jul 07 18:48:19.640709 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640634 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-web-config" (OuterVolumeSpecName: "web-config") pod "e34479f2-1632-4876-bdb9-bda323223879" (UID: "e34479f2-1632-4876-bdb9-bda323223879"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:48:19.640868 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640846 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558"} err="failed to get container status \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": rpc error: code = NotFound desc = could not find container \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": container with ID starting with 06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558 not found: ID does not exist" Jul 07 18:48:19.640868 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.640866 2582 scope.go:117] "RemoveContainer" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" Jul 07 18:48:19.641092 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.641075 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e"} err="failed to get container status \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": rpc error: code = NotFound desc = could not find container \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": container with ID starting with f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e not found: ID does not exist" Jul 07 18:48:19.641132 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.641093 2582 scope.go:117] "RemoveContainer" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" Jul 07 18:48:19.641327 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.641311 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e"} err="failed to get container status \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": rpc error: code = NotFound desc = could not find container \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": container with ID starting with 1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e not found: ID does not exist" Jul 07 18:48:19.641360 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.641330 2582 scope.go:117] "RemoveContainer" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" Jul 07 18:48:19.641545 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.641529 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8"} err="failed to get container status \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": rpc error: code = NotFound desc = could not find container \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": container with ID starting with 8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8 not found: ID does not exist" Jul 07 18:48:19.641593 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.641545 2582 scope.go:117] "RemoveContainer" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" Jul 07 18:48:19.641813 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.641794 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e"} err="failed to get container status \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": rpc error: code = NotFound desc = could not find container \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": container with ID starting with 75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e not found: ID does not exist" Jul 07 18:48:19.641893 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.641814 2582 scope.go:117] "RemoveContainer" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" Jul 07 18:48:19.642049 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642032 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc"} err="failed to get container status \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": rpc error: code = NotFound desc = could not find container \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": container with ID starting with b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc not found: ID does not exist" Jul 07 18:48:19.642092 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642050 2582 scope.go:117] "RemoveContainer" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" Jul 07 18:48:19.642250 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642235 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c"} err="failed to get container status \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": rpc error: code = NotFound desc = could not find container \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": container with ID starting with 24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c not found: ID does not exist" Jul 07 18:48:19.642250 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642249 2582 scope.go:117] "RemoveContainer" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" Jul 07 18:48:19.642425 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642411 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558"} err="failed to get container status \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": rpc error: code = NotFound desc = could not find container \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": container with ID starting with 06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558 not found: ID does not exist" Jul 07 18:48:19.642467 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642425 2582 scope.go:117] "RemoveContainer" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" Jul 07 18:48:19.642659 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642628 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e"} err="failed to get container status \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": rpc error: code = NotFound desc = could not find container \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": container with ID starting with f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e not found: ID does not exist" Jul 07 18:48:19.642702 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642663 2582 scope.go:117] "RemoveContainer" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" Jul 07 18:48:19.642893 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642876 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e"} err="failed to get container status \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": rpc error: code = NotFound desc = could not find container \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": container with ID starting with 1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e not found: ID does not exist" Jul 07 18:48:19.642961 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.642895 2582 scope.go:117] "RemoveContainer" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" Jul 07 18:48:19.643118 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643100 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8"} err="failed to get container status \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": rpc error: code = NotFound desc = could not find container \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": container with ID starting with 8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8 not found: ID does not exist" Jul 07 18:48:19.643162 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643120 2582 scope.go:117] "RemoveContainer" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" Jul 07 18:48:19.643307 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643290 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e"} err="failed to get container status \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": rpc error: code = NotFound desc = could not find container \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": container with ID starting with 75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e not found: ID does not exist" Jul 07 18:48:19.643372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643308 2582 scope.go:117] "RemoveContainer" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" Jul 07 18:48:19.643512 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643498 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc"} err="failed to get container status \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": rpc error: code = NotFound desc = could not find container \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": container with ID starting with b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc not found: ID does not exist" Jul 07 18:48:19.643557 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643512 2582 scope.go:117] "RemoveContainer" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" Jul 07 18:48:19.643705 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643688 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c"} err="failed to get container status \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": rpc error: code = NotFound desc = could not find container \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": container with ID starting with 24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c not found: ID does not exist" Jul 07 18:48:19.643766 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643706 2582 scope.go:117] "RemoveContainer" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" Jul 07 18:48:19.643902 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643887 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558"} err="failed to get container status \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": rpc error: code = NotFound desc = could not find container \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": container with ID starting with 06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558 not found: ID does not exist" Jul 07 18:48:19.643950 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.643902 2582 scope.go:117] "RemoveContainer" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" Jul 07 18:48:19.644097 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644077 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e"} err="failed to get container status \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": rpc error: code = NotFound desc = could not find container \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": container with ID starting with f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e not found: ID does not exist" Jul 07 18:48:19.644140 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644099 2582 scope.go:117] "RemoveContainer" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" Jul 07 18:48:19.644287 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644270 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e"} err="failed to get container status \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": rpc error: code = NotFound desc = could not find container \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": container with ID starting with 1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e not found: ID does not exist" Jul 07 18:48:19.644330 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644287 2582 scope.go:117] "RemoveContainer" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" Jul 07 18:48:19.644484 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644468 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8"} err="failed to get container status \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": rpc error: code = NotFound desc = could not find container \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": container with ID starting with 8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8 not found: ID does not exist" Jul 07 18:48:19.644555 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644485 2582 scope.go:117] "RemoveContainer" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" Jul 07 18:48:19.644681 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644638 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e"} err="failed to get container status \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": rpc error: code = NotFound desc = could not find container \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": container with ID starting with 75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e not found: ID does not exist" Jul 07 18:48:19.644753 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644683 2582 scope.go:117] "RemoveContainer" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" Jul 07 18:48:19.644897 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644880 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc"} err="failed to get container status \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": rpc error: code = NotFound desc = could not find container \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": container with ID starting with b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc not found: ID does not exist" Jul 07 18:48:19.644964 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.644897 2582 scope.go:117] "RemoveContainer" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" Jul 07 18:48:19.645116 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645095 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c"} err="failed to get container status \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": rpc error: code = NotFound desc = could not find container \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": container with ID starting with 24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c not found: ID does not exist" Jul 07 18:48:19.645163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645118 2582 scope.go:117] "RemoveContainer" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" Jul 07 18:48:19.645309 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645293 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558"} err="failed to get container status \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": rpc error: code = NotFound desc = could not find container \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": container with ID starting with 06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558 not found: ID does not exist" Jul 07 18:48:19.645356 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645309 2582 scope.go:117] "RemoveContainer" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" Jul 07 18:48:19.645509 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645486 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e"} err="failed to get container status \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": rpc error: code = NotFound desc = could not find container \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": container with ID starting with f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e not found: ID does not exist" Jul 07 18:48:19.645509 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645507 2582 scope.go:117] "RemoveContainer" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" Jul 07 18:48:19.645721 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645707 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e"} err="failed to get container status \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": rpc error: code = NotFound desc = could not find container \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": container with ID starting with 1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e not found: ID does not exist" Jul 07 18:48:19.645769 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645722 2582 scope.go:117] "RemoveContainer" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" Jul 07 18:48:19.645937 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645922 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8"} err="failed to get container status \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": rpc error: code = NotFound desc = could not find container \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": container with ID starting with 8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8 not found: ID does not exist" Jul 07 18:48:19.645972 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.645937 2582 scope.go:117] "RemoveContainer" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" Jul 07 18:48:19.646113 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646079 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e"} err="failed to get container status \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": rpc error: code = NotFound desc = could not find container \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": container with ID starting with 75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e not found: ID does not exist" Jul 07 18:48:19.646157 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646114 2582 scope.go:117] "RemoveContainer" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" Jul 07 18:48:19.646254 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646238 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc"} err="failed to get container status \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": rpc error: code = NotFound desc = could not find container \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": container with ID starting with b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc not found: ID does not exist" Jul 07 18:48:19.646254 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646254 2582 scope.go:117] "RemoveContainer" containerID="24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c" Jul 07 18:48:19.646433 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646416 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c"} err="failed to get container status \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": rpc error: code = NotFound desc = could not find container \"24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c\": container with ID starting with 24c9d5aee5b85ff9e93fbd7cbbec8516fa3391b2c0a77e78f9ca250aa488f74c not found: ID does not exist" Jul 07 18:48:19.646474 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646433 2582 scope.go:117] "RemoveContainer" containerID="06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558" Jul 07 18:48:19.646627 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646609 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558"} err="failed to get container status \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": rpc error: code = NotFound desc = could not find container \"06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558\": container with ID starting with 06cb8ff52a6822f9494d2aa16e10645308d1ff3a18cce7591c61db602947c558 not found: ID does not exist" Jul 07 18:48:19.646751 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646628 2582 scope.go:117] "RemoveContainer" containerID="f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e" Jul 07 18:48:19.646851 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646835 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e"} err="failed to get container status \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": rpc error: code = NotFound desc = could not find container \"f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e\": container with ID starting with f1063689fe33526e2ab3e677813d7c8f731efa7571837c07f8839cdc63386a5e not found: ID does not exist" Jul 07 18:48:19.646896 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.646851 2582 scope.go:117] "RemoveContainer" containerID="1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e" Jul 07 18:48:19.647035 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.647015 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e"} err="failed to get container status \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": rpc error: code = NotFound desc = could not find container \"1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e\": container with ID starting with 1549c31b1b521fea0d0f79e445e5c031d135cdede0a131b1416ae1b94bc0d25e not found: ID does not exist" Jul 07 18:48:19.647076 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.647038 2582 scope.go:117] "RemoveContainer" containerID="8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8" Jul 07 18:48:19.647237 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.647220 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8"} err="failed to get container status \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": rpc error: code = NotFound desc = could not find container \"8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8\": container with ID starting with 8d7f0b5cb610e8d2f149e203b42f22bff1b87c7314b0828c49528483a4b1bdf8 not found: ID does not exist" Jul 07 18:48:19.647307 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.647238 2582 scope.go:117] "RemoveContainer" containerID="75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e" Jul 07 18:48:19.647422 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.647408 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e"} err="failed to get container status \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": rpc error: code = NotFound desc = could not find container \"75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e\": container with ID starting with 75c3614895963a4b160e1b6283a78c1f277cf815272c34673ba3da7d60ada08e not found: ID does not exist" Jul 07 18:48:19.647467 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.647423 2582 scope.go:117] "RemoveContainer" containerID="b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc" Jul 07 18:48:19.647610 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.647595 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc"} err="failed to get container status \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": rpc error: code = NotFound desc = could not find container \"b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc\": container with ID starting with b30f42e59dacfca60559de73be5176865955bff5e6547ffc5c64018f4b3e7dfc not found: ID does not exist" Jul 07 18:48:19.726171 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726141 2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726171 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726167 2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-grpc-tls\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726171 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726176 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726185 2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-kube-rbac-proxy\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726194 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726202 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbqpk\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-kube-api-access-zbqpk\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726211 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726220 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-prometheus-k8s-db\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726228 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-web-config\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726237 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726246 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726254 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e34479f2-1632-4876-bdb9-bda323223879-config-out\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726262 2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-metrics-client-certs\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726269 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e34479f2-1632-4876-bdb9-bda323223879-tls-assets\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726277 2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-config\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.726372 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.726285 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e34479f2-1632-4876-bdb9-bda323223879-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:48:19.908181 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.908150 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:48:19.914971 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.914934 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:48:19.944910 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.944882 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:48:19.945214 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945201 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="prom-label-proxy" Jul 07 18:48:19.945266 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945217 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="prom-label-proxy" Jul 07 18:48:19.945266 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945229 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="config-reloader" Jul 07 18:48:19.945266 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945235 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="config-reloader" Jul 07 18:48:19.945266 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945243 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="prometheus" Jul 07 18:48:19.945266 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945247 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="prometheus" Jul 07 18:48:19.945266 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945252 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="config-reloader" Jul 07 18:48:19.945266 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945257 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="config-reloader" Jul 07 18:48:19.945266 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945266 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="init-config-reloader" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945271 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="init-config-reloader" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945277 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy-web" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945283 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy-web" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945293 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="alertmanager" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945298 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="alertmanager" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945305 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945310 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945315 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="init-config-reloader" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945319 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="init-config-reloader" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945325 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="thanos-sidecar" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945329 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="thanos-sidecar" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945335 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy-metric" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945340 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy-metric" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945346 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy-web" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945351 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy-web" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945357 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945362 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945369 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy-thanos" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945374 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy-thanos" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945424 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy-thanos" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945432 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="config-reloader" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945440 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy-metric" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945446 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="prometheus" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945452 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945456 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="alertmanager" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945461 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="prom-label-proxy" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945466 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy-web" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945471 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="kube-rbac-proxy-web" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945477 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="config-reloader" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945481 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34479f2-1632-4876-bdb9-bda323223879" containerName="thanos-sidecar" Jul 07 18:48:19.945505 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.945487 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93c8acb-2161-4317-9c06-af57bc9ec0e4" containerName="kube-rbac-proxy" Jul 07 18:48:19.949780 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.949762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:19.952391 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.952371 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 18:48:19.952498 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.952394 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 18:48:19.952498 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.952371 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 18:48:19.952498 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.952468 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 18:48:19.952827 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.952796 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 18:48:19.952921 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.952840 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 18:48:19.952921 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.952840 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 18:48:19.953140 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.953092 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 18:48:19.953243 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.953226 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 18:48:19.953311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.953271 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 18:48:19.953311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.953277 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 18:48:19.953311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.953300 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-s7gandu7pon0\"" Jul 07 18:48:19.953480 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.953271 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 18:48:19.953480 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.953375 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 18:48:19.956605 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.956580 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-c2g8k\"" Jul 07 18:48:19.956924 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.956908 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 18:48:19.960873 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.960856 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 18:48:19.967793 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:19.967769 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:48:20.027882 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.027860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028002 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.027903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028002 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.027927 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028002 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.027942 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028002 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.027991 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028032 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028050 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028069 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028083 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028137 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028167 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028187 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028202 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028223 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028264 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028311 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028285 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ng7f\" (UniqueName: \"kubernetes.io/projected/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-kube-api-access-7ng7f\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028485 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028326 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028485 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028354 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028485 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028385 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-config\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.028485 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.028400 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.128792 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.128759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-config\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.128792 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.128794 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129010 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.128820 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129010 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.128862 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129010 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.128890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129010 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.128911 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129010 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.128937 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129010 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.128988 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129010 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129008 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129036 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129056 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129087 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129111 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129131 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129153 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129185 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129252 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ng7f\" (UniqueName: \"kubernetes.io/projected/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-kube-api-access-7ng7f\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129286 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129318 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.129897 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129696 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.130152 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.129999 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.130689 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.130289 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.131303 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.130687 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.131303 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.130995 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.132159 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.132130 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-config\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.132265 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.132228 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.132565 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.132516 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.133234 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.132778 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.133234 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.132873 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.133234 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.133115 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.133234 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.133140 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.133529 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.133510 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.134056 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.134015 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.134312 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.134075 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.134850 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.134830 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.134940 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.134915 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.135006 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.134982 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.135500 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.135485 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.141558 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.141507 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ng7f\" (UniqueName: \"kubernetes.io/projected/7e1f9e42-1173-43c2-8a1f-f1cbe42377b7-kube-api-access-7ng7f\") pod \"prometheus-k8s-0\" (UID: \"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.261272 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.261240 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:48:20.404279 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.404219 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:48:20.406598 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:48:20.406572 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1f9e42_1173_43c2_8a1f_f1cbe42377b7.slice/crio-7e750a6fa45717cc526f3395cabca4baf5fdffefb9489b353b82e3b0ba753051 WatchSource:0}: Error finding container 7e750a6fa45717cc526f3395cabca4baf5fdffefb9489b353b82e3b0ba753051: Status 404 returned error can't find the container with id 7e750a6fa45717cc526f3395cabca4baf5fdffefb9489b353b82e3b0ba753051 Jul 07 18:48:20.587511 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.587478 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e34479f2-1632-4876-bdb9-bda323223879" path="/var/lib/kubelet/pods/e34479f2-1632-4876-bdb9-bda323223879/volumes" Jul 07 18:48:20.588737 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.588709 2582 generic.go:356] "Generic (PLEG): container finished" podID="7e1f9e42-1173-43c2-8a1f-f1cbe42377b7" containerID="d741380282bf73f0a57b79f9c91fba73132e2ce98995d0514403258912a2273b" exitCode=0 Jul 07 18:48:20.588851 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.588783 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7","Type":"ContainerDied","Data":"d741380282bf73f0a57b79f9c91fba73132e2ce98995d0514403258912a2273b"} Jul 07 18:48:20.588851 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:20.588841 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7","Type":"ContainerStarted","Data":"7e750a6fa45717cc526f3395cabca4baf5fdffefb9489b353b82e3b0ba753051"} Jul 07 18:48:21.597511 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:21.597476 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7","Type":"ContainerStarted","Data":"268e817ec8337c9bdcb93c1bd310475c013a01caa69e4913dc0bc9d57f60e833"} Jul 07 18:48:21.597987 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:21.597561 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7","Type":"ContainerStarted","Data":"3937b0472cf4cf6a1f692314123422e01f12ad437d44759e64e2ad747171c145"} Jul 07 18:48:21.597987 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:21.597591 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7","Type":"ContainerStarted","Data":"fe56f1bdc6e761a653cae1472d15a63dd93d0d4e27448317aab9e074539dc57e"} Jul 07 18:48:21.597987 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:21.597616 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7","Type":"ContainerStarted","Data":"1b3210065adf11d8ce5d9fc3fdf8c50746ae62ac35b6381abe934c6991bb067c"} Jul 07 18:48:21.597987 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:21.597640 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7","Type":"ContainerStarted","Data":"a098c4953b744254dbf99fa5cdf058ad14caef14e5c7b584d3dfd65276c891b6"} Jul 07 18:48:21.597987 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:21.597688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e1f9e42-1173-43c2-8a1f-f1cbe42377b7","Type":"ContainerStarted","Data":"26480b906eabcfd6df8fdeac3426aeacef0a15c49da31da5c265c64b19437316"} Jul 07 18:48:21.629170 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:21.629115 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.609975963 podStartE2EDuration="2.629094776s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.118813ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:48:21.626532326 +0000 UTC m=+203.651650427" watchObservedRunningTime="2026-07-07 18:48:21.629094776 +0000 UTC m=+203.654212850" Jul 07 18:48:25.261764 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:48:25.261731 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:20.262221 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:20.262140 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:20.278100 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:20.278071 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:20.886243 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:20.886210 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:49:56.077457 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.077427 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x"] Jul 07 18:49:56.079879 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.079862 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.083630 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.083612 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mqw76\"" Jul 07 18:49:56.084211 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.084192 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 18:49:56.084324 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.084276 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 18:49:56.084506 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.084489 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 18:49:56.084840 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.084824 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 18:49:56.113047 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.113020 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x"] Jul 07 18:49:56.152208 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.152177 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.152387 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.152244 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-manifests\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.152387 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.152277 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-webhook-cert\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.152387 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.152298 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qltq\" (UniqueName: \"kubernetes.io/projected/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-kube-api-access-8qltq\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.152516 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.152413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-tmp\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.253844 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.253810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-tmp\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.254012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.253845 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.254012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.253960 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-manifests\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.254012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.253986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-webhook-cert\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.254012 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.254005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qltq\" (UniqueName: \"kubernetes.io/projected/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-kube-api-access-8qltq\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.254377 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.254349 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-manifests\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.256160 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.256141 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-tmp\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.256264 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.256247 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.256365 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.256349 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-webhook-cert\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.272640 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.272614 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qltq\" (UniqueName: \"kubernetes.io/projected/2dfeb259-f3af-4a62-8bee-c03dd68ef2ee-kube-api-access-8qltq\") pod \"opendatahub-operator-controller-manager-545cccdbd8-6xl9x\" (UID: \"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee\") " pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.390100 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.390015 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:49:56.536141 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:56.536119 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x"] Jul 07 18:49:56.538437 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:49:56.538415 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dfeb259_f3af_4a62_8bee_c03dd68ef2ee.slice/crio-f23c20a7fa264030533e9993f9627718f28ba86b3e9bd90ae093ea7f7179cadc WatchSource:0}: Error finding container f23c20a7fa264030533e9993f9627718f28ba86b3e9bd90ae093ea7f7179cadc: Status 404 returned error can't find the container with id f23c20a7fa264030533e9993f9627718f28ba86b3e9bd90ae093ea7f7179cadc Jul 07 18:49:57.023493 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:57.023455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" event={"ID":"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee","Type":"ContainerStarted","Data":"f23c20a7fa264030533e9993f9627718f28ba86b3e9bd90ae093ea7f7179cadc"} Jul 07 18:49:58.529616 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:58.529468 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:49:58.538206 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:58.532464 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:49:58.538206 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:58.536840 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:49:58.540604 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:58.540581 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:49:58.545243 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:49:58.545227 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 18:50:00.046798 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:00.046764 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" event={"ID":"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee","Type":"ContainerStarted","Data":"fc547b053f881f9638cca05f845980a273e67215b527ba5dcb0761c77a81485a"} Jul 07 18:50:01.054158 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:01.054060 2582 generic.go:356] "Generic (PLEG): container finished" podID="2dfeb259-f3af-4a62-8bee-c03dd68ef2ee" containerID="fc547b053f881f9638cca05f845980a273e67215b527ba5dcb0761c77a81485a" exitCode=0 Jul 07 18:50:01.054158 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:01.054117 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" event={"ID":"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee","Type":"ContainerDied","Data":"fc547b053f881f9638cca05f845980a273e67215b527ba5dcb0761c77a81485a"} Jul 07 18:50:01.054813 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:01.054793 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:50:02.061141 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:02.061101 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" event={"ID":"2dfeb259-f3af-4a62-8bee-c03dd68ef2ee","Type":"ContainerStarted","Data":"eb1c6fe48c1a2429566145f1f41277a4a70fba3495b3af3a1c9ad4dc2fe468da"} Jul 07 18:50:02.061636 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:02.061204 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:50:02.086707 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:02.086625 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" podStartSLOduration=1.6201687740000001 podStartE2EDuration="6.086607607s" totalImagesPullingTime="3.382815076s" totalInitContainerRuntime="1.083623757s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:02.084727897 +0000 UTC m=+304.109845987" watchObservedRunningTime="2026-07-07 18:50:02.086607607 +0000 UTC m=+304.111725682" Jul 07 18:50:13.067748 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:13.067718 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-545cccdbd8-6xl9x" Jul 07 18:50:17.061705 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.061670 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh"] Jul 07 18:50:17.072571 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.072543 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.075056 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.075034 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-f86wh\"" Jul 07 18:50:17.075198 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.075035 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 18:50:17.075872 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.075853 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 18:50:17.075988 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.075858 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 18:50:17.076111 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.076096 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 18:50:17.082684 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.082222 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh"] Jul 07 18:50:17.142901 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.142870 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tls-certs\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.143072 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.142922 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg57k\" (UniqueName: \"kubernetes.io/projected/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-kube-api-access-mg57k\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.143072 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.143035 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tmp\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.244547 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.244515 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tmp\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.244765 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.244609 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tls-certs\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.244765 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.244664 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg57k\" (UniqueName: \"kubernetes.io/projected/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-kube-api-access-mg57k\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.245068 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:50:17.245035 2582 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 07 18:50:17.245209 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:50:17.245144 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tls-certs podName:3d59c838-e730-4eed-aa2f-f8d6fe323a4e nodeName:}" failed. No retries permitted until 2026-07-07 18:50:17.745123161 +0000 UTC m=+319.770241223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tls-certs") pod "kube-auth-proxy-d58d67c5c-rgvhh" (UID: "3d59c838-e730-4eed-aa2f-f8d6fe323a4e") : secret "kube-auth-proxy-tls" not found Jul 07 18:50:17.247362 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.247339 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tmp\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.254609 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.254584 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg57k\" (UniqueName: \"kubernetes.io/projected/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-kube-api-access-mg57k\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.750413 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.750375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tls-certs\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.752935 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.752910 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d59c838-e730-4eed-aa2f-f8d6fe323a4e-tls-certs\") pod \"kube-auth-proxy-d58d67c5c-rgvhh\" (UID: \"3d59c838-e730-4eed-aa2f-f8d6fe323a4e\") " pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:17.984968 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:17.984922 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" Jul 07 18:50:18.141323 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:50:18.141293 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d59c838_e730_4eed_aa2f_f8d6fe323a4e.slice/crio-802f03eba4620d0e75a718f82b9057c0c1e33a3595d0a062bf7504bfd26ccf79 WatchSource:0}: Error finding container 802f03eba4620d0e75a718f82b9057c0c1e33a3595d0a062bf7504bfd26ccf79: Status 404 returned error can't find the container with id 802f03eba4620d0e75a718f82b9057c0c1e33a3595d0a062bf7504bfd26ccf79 Jul 07 18:50:18.149946 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:18.149924 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh"] Jul 07 18:50:19.140347 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:19.140310 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" event={"ID":"3d59c838-e730-4eed-aa2f-f8d6fe323a4e","Type":"ContainerStarted","Data":"802f03eba4620d0e75a718f82b9057c0c1e33a3595d0a062bf7504bfd26ccf79"} Jul 07 18:50:21.831004 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.830970 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk"] Jul 07 18:50:21.834060 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.834042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.837205 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.836925 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 18:50:21.837205 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.836991 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6rjlq\"" Jul 07 18:50:21.845951 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.845924 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk"] Jul 07 18:50:21.890438 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.890438 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890439 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.890690 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890459 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.890690 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890512 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fe74e65-fce9-476c-9abd-0518cab2c81b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.890690 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890574 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.890690 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890609 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7hdn\" (UniqueName: \"kubernetes.io/projected/3fe74e65-fce9-476c-9abd-0518cab2c81b-kube-api-access-t7hdn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.890847 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.890847 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890739 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.890847 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.890775 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.991708 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.991676 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.991708 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.991712 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.991965 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.991730 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fe74e65-fce9-476c-9abd-0518cab2c81b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.991965 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.991859 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.991965 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.991915 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7hdn\" (UniqueName: \"kubernetes.io/projected/3fe74e65-fce9-476c-9abd-0518cab2c81b-kube-api-access-t7hdn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992136 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.991971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992136 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.992040 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992136 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.992125 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992277 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.992179 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992334 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.992288 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992389 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.992363 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992493 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.992475 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fe74e65-fce9-476c-9abd-0518cab2c81b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992667 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.992626 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.992794 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.992769 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.994622 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.994603 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:21.994786 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:21.994769 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:22.006630 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:22.006603 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fe74e65-fce9-476c-9abd-0518cab2c81b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:22.007870 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:22.007853 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7hdn\" (UniqueName: \"kubernetes.io/projected/3fe74e65-fce9-476c-9abd-0518cab2c81b-kube-api-access-t7hdn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xx2vk\" (UID: \"3fe74e65-fce9-476c-9abd-0518cab2c81b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:22.148131 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:22.148045 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:22.157511 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:22.157475 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" event={"ID":"3d59c838-e730-4eed-aa2f-f8d6fe323a4e","Type":"ContainerStarted","Data":"fa014b777502c3ea2e67d91ef3136de48b084245c4884e2425db8d997edad017"} Jul 07 18:50:22.177918 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:22.177861 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-d58d67c5c-rgvhh" podStartSLOduration=2.158113769 podStartE2EDuration="5.177840141s" totalImagesPullingTime="3.019726372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:22.176760088 +0000 UTC m=+324.201878161" watchObservedRunningTime="2026-07-07 18:50:22.177840141 +0000 UTC m=+324.202958215" Jul 07 18:50:22.307191 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:22.307164 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk"] Jul 07 18:50:22.308295 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:50:22.308265 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe74e65_fce9_476c_9abd_0518cab2c81b.slice/crio-797a68cb005723c7b7c58d8b66e39400da59d32b237e48431b41211d914f8059 WatchSource:0}: Error finding container 797a68cb005723c7b7c58d8b66e39400da59d32b237e48431b41211d914f8059: Status 404 returned error can't find the container with id 797a68cb005723c7b7c58d8b66e39400da59d32b237e48431b41211d914f8059 Jul 07 18:50:23.166387 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.166345 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" event={"ID":"3fe74e65-fce9-476c-9abd-0518cab2c81b","Type":"ContainerStarted","Data":"797a68cb005723c7b7c58d8b66e39400da59d32b237e48431b41211d914f8059"} Jul 07 18:50:23.750568 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.750534 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv"] Jul 07 18:50:23.754478 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.754453 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.758139 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.758114 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-znrn5\"" Jul 07 18:50:23.758282 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.758173 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 18:50:23.758777 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.758550 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 18:50:23.758777 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.758604 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 18:50:23.758777 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.758676 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:50:23.758777 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.758683 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 18:50:23.772063 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.772013 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv"] Jul 07 18:50:23.811980 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.811941 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-metrics-cert\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.811980 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.811995 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-cert\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.812232 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.812037 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jllbf\" (UniqueName: \"kubernetes.io/projected/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-kube-api-access-jllbf\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.812232 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.812133 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-manager-config\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.913114 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.913073 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jllbf\" (UniqueName: \"kubernetes.io/projected/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-kube-api-access-jllbf\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.913295 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.913197 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-manager-config\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.913366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.913266 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-metrics-cert\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.913366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.913322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-cert\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.914153 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.914068 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-manager-config\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.916307 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.916274 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-cert\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.916499 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.916476 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-metrics-cert\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:23.924756 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:23.924709 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jllbf\" (UniqueName: \"kubernetes.io/projected/c9d1da0e-66c0-422d-935e-ac6a2f7af12e-kube-api-access-jllbf\") pod \"lws-controller-manager-66b96648c9-h4wgv\" (UID: \"c9d1da0e-66c0-422d-935e-ac6a2f7af12e\") " pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:24.067306 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:24.067273 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:25.048352 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:25.048063 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 18:50:25.048352 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:25.048175 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 18:50:25.048352 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:25.048217 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 07 18:50:25.146799 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:25.146708 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv"] Jul 07 18:50:25.149143 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:50:25.149114 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d1da0e_66c0_422d_935e_ac6a2f7af12e.slice/crio-ad92110b5efadff8ecfea1c70d79bc48f86ed3a30763666d249f7b19edb79f89 WatchSource:0}: Error finding container ad92110b5efadff8ecfea1c70d79bc48f86ed3a30763666d249f7b19edb79f89: Status 404 returned error can't find the container with id ad92110b5efadff8ecfea1c70d79bc48f86ed3a30763666d249f7b19edb79f89 Jul 07 18:50:25.179273 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:25.179121 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" event={"ID":"3fe74e65-fce9-476c-9abd-0518cab2c81b","Type":"ContainerStarted","Data":"9d392184c03a763b9be92a912c883e495f365cc07344b0446b16c4049b5ec446"} Jul 07 18:50:25.180468 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:25.180443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" event={"ID":"c9d1da0e-66c0-422d-935e-ac6a2f7af12e","Type":"ContainerStarted","Data":"ad92110b5efadff8ecfea1c70d79bc48f86ed3a30763666d249f7b19edb79f89"} Jul 07 18:50:25.207748 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:25.207693 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" podStartSLOduration=1.470264623 podStartE2EDuration="4.207676582s" totalImagesPullingTime="2.737411959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:25.205443264 +0000 UTC m=+327.230561351" watchObservedRunningTime="2026-07-07 18:50:25.207676582 +0000 UTC m=+327.232794654" Jul 07 18:50:26.148506 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:26.148462 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:26.153384 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:26.153356 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:26.185329 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:26.185299 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:26.186159 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:26.186137 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xx2vk" Jul 07 18:50:32.221240 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:32.221205 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" event={"ID":"c9d1da0e-66c0-422d-935e-ac6a2f7af12e","Type":"ContainerStarted","Data":"9061cf867574391b4ae2284d0204270a8f4b02580a00af08d929872588d9216a"} Jul 07 18:50:32.221711 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:32.221289 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:50:32.255022 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:32.254929 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" podStartSLOduration=2.613174167 podStartE2EDuration="9.254911617s" totalImagesPullingTime="6.64173745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:32.25419109 +0000 UTC m=+334.279309163" watchObservedRunningTime="2026-07-07 18:50:32.254911617 +0000 UTC m=+334.280029691" Jul 07 18:50:43.228671 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:50:43.228617 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66b96648c9-h4wgv" Jul 07 18:52:00.532891 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.532858 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q"] Jul 07 18:52:00.538240 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.538221 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" Jul 07 18:52:00.541174 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.541154 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 18:52:00.541938 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.541916 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-n4h2t\"" Jul 07 18:52:00.542053 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.541946 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 18:52:00.542053 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.541948 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 18:52:00.550043 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.550024 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q"] Jul 07 18:52:00.566374 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.566349 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tc8h\" (UniqueName: \"kubernetes.io/projected/7362ab14-47ce-499f-9375-e8f1d48f82b2-kube-api-access-6tc8h\") pod \"dns-operator-controller-manager-864dd78c4c-zwl8q\" (UID: \"7362ab14-47ce-499f-9375-e8f1d48f82b2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" Jul 07 18:52:00.666844 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.666811 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tc8h\" (UniqueName: \"kubernetes.io/projected/7362ab14-47ce-499f-9375-e8f1d48f82b2-kube-api-access-6tc8h\") pod \"dns-operator-controller-manager-864dd78c4c-zwl8q\" (UID: \"7362ab14-47ce-499f-9375-e8f1d48f82b2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" Jul 07 18:52:00.679195 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.679171 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tc8h\" (UniqueName: \"kubernetes.io/projected/7362ab14-47ce-499f-9375-e8f1d48f82b2-kube-api-access-6tc8h\") pod \"dns-operator-controller-manager-864dd78c4c-zwl8q\" (UID: \"7362ab14-47ce-499f-9375-e8f1d48f82b2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" Jul 07 18:52:00.848416 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.848380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" Jul 07 18:52:00.990256 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:00.990221 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q"] Jul 07 18:52:00.993601 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:52:00.993570 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7362ab14_47ce_499f_9375_e8f1d48f82b2.slice/crio-a2410c2075e6206754cfd5cbc66c039dda5356954ec428813250b52f4f3d1123 WatchSource:0}: Error finding container a2410c2075e6206754cfd5cbc66c039dda5356954ec428813250b52f4f3d1123: Status 404 returned error can't find the container with id a2410c2075e6206754cfd5cbc66c039dda5356954ec428813250b52f4f3d1123 Jul 07 18:52:01.633044 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:01.633010 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" event={"ID":"7362ab14-47ce-499f-9375-e8f1d48f82b2","Type":"ContainerStarted","Data":"a2410c2075e6206754cfd5cbc66c039dda5356954ec428813250b52f4f3d1123"} Jul 07 18:52:03.644821 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:03.644783 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" event={"ID":"7362ab14-47ce-499f-9375-e8f1d48f82b2","Type":"ContainerStarted","Data":"e4353bd36fdcaaaa169204bdb6ed9a0ca10ff41550cc4c6dc301900c31cb581b"} Jul 07 18:52:03.645190 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:03.644913 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" Jul 07 18:52:03.668192 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:03.668149 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" podStartSLOduration=1.10939134 podStartE2EDuration="3.668106138s" totalImagesPullingTime="2.558714798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:52:03.667277686 +0000 UTC m=+425.692395778" watchObservedRunningTime="2026-07-07 18:52:03.668106138 +0000 UTC m=+425.693224211" Jul 07 18:52:04.784023 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.783990 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc"] Jul 07 18:52:04.787824 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.787799 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:04.790312 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.790284 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-95977\"" Jul 07 18:52:04.790620 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.790600 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 18:52:04.790799 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.790778 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 18:52:04.800489 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.800459 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc"] Jul 07 18:52:04.906252 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.906215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86b4d633-a357-4fc0-8699-f24e76b2e387-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:04.906437 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.906255 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/86b4d633-a357-4fc0-8699-f24e76b2e387-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:04.906437 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:04.906293 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdbtd\" (UniqueName: \"kubernetes.io/projected/86b4d633-a357-4fc0-8699-f24e76b2e387-kube-api-access-gdbtd\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.007537 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.007497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86b4d633-a357-4fc0-8699-f24e76b2e387-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.007537 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.007542 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/86b4d633-a357-4fc0-8699-f24e76b2e387-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.007828 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.007577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdbtd\" (UniqueName: \"kubernetes.io/projected/86b4d633-a357-4fc0-8699-f24e76b2e387-kube-api-access-gdbtd\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.007828 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:52:05.007751 2582 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 07 18:52:05.007935 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:52:05.007835 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86b4d633-a357-4fc0-8699-f24e76b2e387-plugin-serving-cert podName:86b4d633-a357-4fc0-8699-f24e76b2e387 nodeName:}" failed. No retries permitted until 2026-07-07 18:52:05.507811549 +0000 UTC m=+427.532929604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/86b4d633-a357-4fc0-8699-f24e76b2e387-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-wc2gc" (UID: "86b4d633-a357-4fc0-8699-f24e76b2e387") : secret "plugin-serving-cert" not found Jul 07 18:52:05.008237 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.008216 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/86b4d633-a357-4fc0-8699-f24e76b2e387-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.017239 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.017213 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdbtd\" (UniqueName: \"kubernetes.io/projected/86b4d633-a357-4fc0-8699-f24e76b2e387-kube-api-access-gdbtd\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.512330 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.512293 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/86b4d633-a357-4fc0-8699-f24e76b2e387-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.514909 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.514885 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/86b4d633-a357-4fc0-8699-f24e76b2e387-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wc2gc\" (UID: \"86b4d633-a357-4fc0-8699-f24e76b2e387\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.715027 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.714997 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" Jul 07 18:52:05.852625 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:05.852601 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc"] Jul 07 18:52:05.854751 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:52:05.854724 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86b4d633_a357_4fc0_8699_f24e76b2e387.slice/crio-551a2e799d363a46949736f0df7bc04a25bab6795934a5329770e8541ec1ef30 WatchSource:0}: Error finding container 551a2e799d363a46949736f0df7bc04a25bab6795934a5329770e8541ec1ef30: Status 404 returned error can't find the container with id 551a2e799d363a46949736f0df7bc04a25bab6795934a5329770e8541ec1ef30 Jul 07 18:52:06.659828 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:06.659795 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" event={"ID":"86b4d633-a357-4fc0-8699-f24e76b2e387","Type":"ContainerStarted","Data":"551a2e799d363a46949736f0df7bc04a25bab6795934a5329770e8541ec1ef30"} Jul 07 18:52:14.652325 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:14.652293 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zwl8q" Jul 07 18:52:15.652433 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:15.652399 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk"] Jul 07 18:52:15.656130 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:15.656112 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" Jul 07 18:52:15.658784 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:15.658765 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fhxm4\"" Jul 07 18:52:15.672473 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:15.672452 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk"] Jul 07 18:52:15.712895 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:15.712863 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdbjx\" (UniqueName: \"kubernetes.io/projected/b17f25e6-7e6f-4a70-8117-5139268e32b2-kube-api-access-fdbjx\") pod \"limitador-operator-controller-manager-859d7b78bd-xthgk\" (UID: \"b17f25e6-7e6f-4a70-8117-5139268e32b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" Jul 07 18:52:15.813791 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:15.813756 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdbjx\" (UniqueName: \"kubernetes.io/projected/b17f25e6-7e6f-4a70-8117-5139268e32b2-kube-api-access-fdbjx\") pod \"limitador-operator-controller-manager-859d7b78bd-xthgk\" (UID: \"b17f25e6-7e6f-4a70-8117-5139268e32b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" Jul 07 18:52:15.827799 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:15.827767 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdbjx\" (UniqueName: \"kubernetes.io/projected/b17f25e6-7e6f-4a70-8117-5139268e32b2-kube-api-access-fdbjx\") pod \"limitador-operator-controller-manager-859d7b78bd-xthgk\" (UID: \"b17f25e6-7e6f-4a70-8117-5139268e32b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" Jul 07 18:52:15.967196 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:15.967115 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" Jul 07 18:52:16.123802 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:16.123779 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk"] Jul 07 18:52:16.126535 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:52:16.126505 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb17f25e6_7e6f_4a70_8117_5139268e32b2.slice/crio-18e5fab4d718e14da029c6bf33b0daa93025e4da55ee2e17e2d72768e7999ee3 WatchSource:0}: Error finding container 18e5fab4d718e14da029c6bf33b0daa93025e4da55ee2e17e2d72768e7999ee3: Status 404 returned error can't find the container with id 18e5fab4d718e14da029c6bf33b0daa93025e4da55ee2e17e2d72768e7999ee3 Jul 07 18:52:16.718328 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:16.718287 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" event={"ID":"b17f25e6-7e6f-4a70-8117-5139268e32b2","Type":"ContainerStarted","Data":"18e5fab4d718e14da029c6bf33b0daa93025e4da55ee2e17e2d72768e7999ee3"} Jul 07 18:52:31.801554 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:31.801511 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" event={"ID":"86b4d633-a357-4fc0-8699-f24e76b2e387","Type":"ContainerStarted","Data":"f366d4e5799c20870a832b5a36670fed403ed21bee1bc9ec4b6d69549f9fd828"} Jul 07 18:52:31.821163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:31.821115 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wc2gc" podStartSLOduration=2.680287696 podStartE2EDuration="27.821095195s" totalImagesPullingTime="25.140807499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:52:31.818786676 +0000 UTC m=+453.843904761" watchObservedRunningTime="2026-07-07 18:52:31.821095195 +0000 UTC m=+453.846213269" Jul 07 18:52:32.817512 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:32.817470 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" event={"ID":"b17f25e6-7e6f-4a70-8117-5139268e32b2","Type":"ContainerStarted","Data":"9e926debdbf89337ebab951ba0dc53a3c025c69a9df013eb920affb91a0a4c70"} Jul 07 18:52:32.817975 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:32.817620 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" Jul 07 18:52:32.838402 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:32.838357 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" podStartSLOduration=1.740372341 podStartE2EDuration="17.83833525s" totalImagesPullingTime="16.097962909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:52:32.836225269 +0000 UTC m=+454.861343342" watchObservedRunningTime="2026-07-07 18:52:32.83833525 +0000 UTC m=+454.863453324" Jul 07 18:52:43.825423 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:52:43.825392 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xthgk" Jul 07 18:53:05.086031 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.085998 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-x945n"] Jul 07 18:53:05.151199 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.151163 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x945n"] Jul 07 18:53:05.151361 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.151255 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x945n" Jul 07 18:53:05.153879 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.153860 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-v8gnw\"" Jul 07 18:53:05.267520 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.267487 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k44cf\" (UniqueName: \"kubernetes.io/projected/1ead2935-0f77-4b43-83ed-a928cb06f6e1-kube-api-access-k44cf\") pod \"authorino-75d556bd77-x945n\" (UID: \"1ead2935-0f77-4b43-83ed-a928cb06f6e1\") " pod="kuadrant-system/authorino-75d556bd77-x945n" Jul 07 18:53:05.368743 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.368656 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k44cf\" (UniqueName: \"kubernetes.io/projected/1ead2935-0f77-4b43-83ed-a928cb06f6e1-kube-api-access-k44cf\") pod \"authorino-75d556bd77-x945n\" (UID: \"1ead2935-0f77-4b43-83ed-a928cb06f6e1\") " pod="kuadrant-system/authorino-75d556bd77-x945n" Jul 07 18:53:05.382603 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.382569 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k44cf\" (UniqueName: \"kubernetes.io/projected/1ead2935-0f77-4b43-83ed-a928cb06f6e1-kube-api-access-k44cf\") pod \"authorino-75d556bd77-x945n\" (UID: \"1ead2935-0f77-4b43-83ed-a928cb06f6e1\") " pod="kuadrant-system/authorino-75d556bd77-x945n" Jul 07 18:53:05.460350 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.460314 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x945n" Jul 07 18:53:05.611786 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.611759 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x945n"] Jul 07 18:53:05.614832 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:53:05.614806 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ead2935_0f77_4b43_83ed_a928cb06f6e1.slice/crio-24b2b1020b6bb88b81aa4ecd4a7311a1661e9d1e71a74f8422e2b621664a8018 WatchSource:0}: Error finding container 24b2b1020b6bb88b81aa4ecd4a7311a1661e9d1e71a74f8422e2b621664a8018: Status 404 returned error can't find the container with id 24b2b1020b6bb88b81aa4ecd4a7311a1661e9d1e71a74f8422e2b621664a8018 Jul 07 18:53:05.981718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:05.981679 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x945n" event={"ID":"1ead2935-0f77-4b43-83ed-a928cb06f6e1","Type":"ContainerStarted","Data":"24b2b1020b6bb88b81aa4ecd4a7311a1661e9d1e71a74f8422e2b621664a8018"} Jul 07 18:53:09.000345 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:09.000312 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x945n" event={"ID":"1ead2935-0f77-4b43-83ed-a928cb06f6e1","Type":"ContainerStarted","Data":"bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef"} Jul 07 18:53:09.038102 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:53:09.038049 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-x945n" podStartSLOduration=0.86201024 podStartE2EDuration="4.038030674s" totalImagesPullingTime="3.176020434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:53:09.036056016 +0000 UTC m=+491.061174088" watchObservedRunningTime="2026-07-07 18:53:09.038030674 +0000 UTC m=+491.063148746" Jul 07 18:54:04.401028 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:04.400935 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x945n"] Jul 07 18:54:04.401553 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:04.401158 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-x945n" podUID="1ead2935-0f77-4b43-83ed-a928cb06f6e1" containerName="authorino" containerID="cri-o://bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef" gracePeriod=30 Jul 07 18:54:04.646349 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:04.646328 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x945n" Jul 07 18:54:04.785680 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:04.785594 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k44cf\" (UniqueName: \"kubernetes.io/projected/1ead2935-0f77-4b43-83ed-a928cb06f6e1-kube-api-access-k44cf\") pod \"1ead2935-0f77-4b43-83ed-a928cb06f6e1\" (UID: \"1ead2935-0f77-4b43-83ed-a928cb06f6e1\") " Jul 07 18:54:04.787906 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:04.787878 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ead2935-0f77-4b43-83ed-a928cb06f6e1-kube-api-access-k44cf" (OuterVolumeSpecName: "kube-api-access-k44cf") pod "1ead2935-0f77-4b43-83ed-a928cb06f6e1" (UID: "1ead2935-0f77-4b43-83ed-a928cb06f6e1"). InnerVolumeSpecName "kube-api-access-k44cf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:54:04.886793 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:04.886759 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k44cf\" (UniqueName: \"kubernetes.io/projected/1ead2935-0f77-4b43-83ed-a928cb06f6e1-kube-api-access-k44cf\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 18:54:05.268065 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.268031 2582 generic.go:356] "Generic (PLEG): container finished" podID="1ead2935-0f77-4b43-83ed-a928cb06f6e1" containerID="bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef" exitCode=0 Jul 07 18:54:05.268225 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.268074 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x945n" event={"ID":"1ead2935-0f77-4b43-83ed-a928cb06f6e1","Type":"ContainerDied","Data":"bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef"} Jul 07 18:54:05.268225 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.268092 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x945n" Jul 07 18:54:05.268225 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.268107 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x945n" event={"ID":"1ead2935-0f77-4b43-83ed-a928cb06f6e1","Type":"ContainerDied","Data":"24b2b1020b6bb88b81aa4ecd4a7311a1661e9d1e71a74f8422e2b621664a8018"} Jul 07 18:54:05.268225 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.268124 2582 scope.go:117] "RemoveContainer" containerID="bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef" Jul 07 18:54:05.277494 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.277474 2582 scope.go:117] "RemoveContainer" containerID="bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef" Jul 07 18:54:05.277824 ip-10-0-141-163 kubenswrapper[2582]: E0707 18:54:05.277798 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef\": container with ID starting with bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef not found: ID does not exist" containerID="bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef" Jul 07 18:54:05.277903 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.277836 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef"} err="failed to get container status \"bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef\": rpc error: code = NotFound desc = could not find container \"bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef\": container with ID starting with bbb623b44412a64f9f2decd4b751814322cacaa10c09b11de4ba2ae8e2a971ef not found: ID does not exist" Jul 07 18:54:05.292147 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.292126 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x945n"] Jul 07 18:54:05.298671 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:05.298634 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x945n"] Jul 07 18:54:06.585361 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:06.585330 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ead2935-0f77-4b43-83ed-a928cb06f6e1" path="/var/lib/kubelet/pods/1ead2935-0f77-4b43-83ed-a928cb06f6e1/volumes" Jul 07 18:54:45.746277 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.746239 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cvxvt"] Jul 07 18:54:45.746709 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.746687 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ead2935-0f77-4b43-83ed-a928cb06f6e1" containerName="authorino" Jul 07 18:54:45.746709 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.746699 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ead2935-0f77-4b43-83ed-a928cb06f6e1" containerName="authorino" Jul 07 18:54:45.746785 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.746781 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ead2935-0f77-4b43-83ed-a928cb06f6e1" containerName="authorino" Jul 07 18:54:45.749741 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.749727 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:45.752120 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.752097 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jrnck\"" Jul 07 18:54:45.752246 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.752129 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 18:54:45.760316 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.760286 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cvxvt"] Jul 07 18:54:45.852894 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.852858 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z94xs\" (UniqueName: \"kubernetes.io/projected/47ee66d0-9c2c-40c8-82f1-0509753fe20f-kube-api-access-z94xs\") pod \"payload-processing-58f86c7798-cvxvt\" (UID: \"47ee66d0-9c2c-40c8-82f1-0509753fe20f\") " pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:45.853114 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.852905 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47ee66d0-9c2c-40c8-82f1-0509753fe20f-plugins-config-volume\") pod \"payload-processing-58f86c7798-cvxvt\" (UID: \"47ee66d0-9c2c-40c8-82f1-0509753fe20f\") " pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:45.953955 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.953914 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47ee66d0-9c2c-40c8-82f1-0509753fe20f-plugins-config-volume\") pod \"payload-processing-58f86c7798-cvxvt\" (UID: \"47ee66d0-9c2c-40c8-82f1-0509753fe20f\") " pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:45.954134 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.954077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z94xs\" (UniqueName: \"kubernetes.io/projected/47ee66d0-9c2c-40c8-82f1-0509753fe20f-kube-api-access-z94xs\") pod \"payload-processing-58f86c7798-cvxvt\" (UID: \"47ee66d0-9c2c-40c8-82f1-0509753fe20f\") " pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:45.954705 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.954685 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47ee66d0-9c2c-40c8-82f1-0509753fe20f-plugins-config-volume\") pod \"payload-processing-58f86c7798-cvxvt\" (UID: \"47ee66d0-9c2c-40c8-82f1-0509753fe20f\") " pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:45.974766 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:45.974744 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z94xs\" (UniqueName: \"kubernetes.io/projected/47ee66d0-9c2c-40c8-82f1-0509753fe20f-kube-api-access-z94xs\") pod \"payload-processing-58f86c7798-cvxvt\" (UID: \"47ee66d0-9c2c-40c8-82f1-0509753fe20f\") " pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:46.060549 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:46.060522 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:46.200350 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:46.200318 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cvxvt"] Jul 07 18:54:46.203728 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:54:46.203704 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ee66d0_9c2c_40c8_82f1_0509753fe20f.slice/crio-9bae4e848fb7c79d03dffa23502b2e0dfb7c441daa803a3bb93ed6f3a63c969c WatchSource:0}: Error finding container 9bae4e848fb7c79d03dffa23502b2e0dfb7c441daa803a3bb93ed6f3a63c969c: Status 404 returned error can't find the container with id 9bae4e848fb7c79d03dffa23502b2e0dfb7c441daa803a3bb93ed6f3a63c969c Jul 07 18:54:46.478836 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:46.478750 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" event={"ID":"47ee66d0-9c2c-40c8-82f1-0509753fe20f","Type":"ContainerStarted","Data":"9bae4e848fb7c79d03dffa23502b2e0dfb7c441daa803a3bb93ed6f3a63c969c"} Jul 07 18:54:48.489889 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:48.489853 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" event={"ID":"47ee66d0-9c2c-40c8-82f1-0509753fe20f","Type":"ContainerStarted","Data":"2124770d981c3417903215a07c1d0228a206820cc770ee94eac7c030fb534a54"} Jul 07 18:54:48.490335 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:48.489955 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:54:48.507974 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:48.507916 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" podStartSLOduration=2.072985429 podStartE2EDuration="3.507897205s" totalImagesPullingTime="1.434911776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:48.507844648 +0000 UTC m=+590.532962752" watchObservedRunningTime="2026-07-07 18:54:48.507897205 +0000 UTC m=+590.533015280" Jul 07 18:54:58.563907 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:58.563871 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:54:58.572762 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:58.568590 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:54:58.572762 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:58.570555 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:54:58.576112 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:58.576095 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:54:59.497511 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:54:59.497485 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-cvxvt" Jul 07 18:55:38.027065 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.026993 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw"] Jul 07 18:55:38.030754 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.030737 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.035247 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.035227 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 18:55:38.036284 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.036258 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 18:55:38.036284 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.036275 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 18:55:38.036457 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.036425 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pblp2\"" Jul 07 18:55:38.052245 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.052223 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw"] Jul 07 18:55:38.114060 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.114026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.114236 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.114068 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.114236 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.114169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.114236 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.114207 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.114366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.114242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.114366 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.114294 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq52r\" (UniqueName: \"kubernetes.io/projected/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-kube-api-access-fq52r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.215538 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.215502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.215538 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.215539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.215815 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.215705 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.215815 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.215785 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq52r\" (UniqueName: \"kubernetes.io/projected/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-kube-api-access-fq52r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.215920 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.215826 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.215920 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.215871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.215920 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.215913 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.216112 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.216089 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.216256 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.216237 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.218082 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.218063 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.218257 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.218240 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.224614 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.224593 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq52r\" (UniqueName: \"kubernetes.io/projected/3d6b6d38-93b2-460f-bbe7-db3cbd2c0089-kube-api-access-fq52r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw\" (UID: \"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.341141 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.341114 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:38.478710 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.478688 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw"] Jul 07 18:55:38.481028 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:55:38.480999 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d6b6d38_93b2_460f_bbe7_db3cbd2c0089.slice/crio-4394a774aaec6325be2bf966fdf46beef6ca9f35c86900d214cecf4bf1c6018a WatchSource:0}: Error finding container 4394a774aaec6325be2bf966fdf46beef6ca9f35c86900d214cecf4bf1c6018a: Status 404 returned error can't find the container with id 4394a774aaec6325be2bf966fdf46beef6ca9f35c86900d214cecf4bf1c6018a Jul 07 18:55:38.482777 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.482761 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:55:38.744148 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:38.744067 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" event={"ID":"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089","Type":"ContainerStarted","Data":"4394a774aaec6325be2bf966fdf46beef6ca9f35c86900d214cecf4bf1c6018a"} Jul 07 18:55:40.754605 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:40.754571 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" event={"ID":"3d6b6d38-93b2-460f-bbe7-db3cbd2c0089","Type":"ContainerStarted","Data":"a4b5e43e25edd56a8a396fbc20379a14d43c3dce6fde0cd35656f2e3a4a9f727"} Jul 07 18:55:40.865530 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:40.865479 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" podStartSLOduration=1.114896666 podStartE2EDuration="2.865462363s" totalImagesPullingTime="1.750565697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:55:40.864852743 +0000 UTC m=+642.889970815" watchObservedRunningTime="2026-07-07 18:55:40.865462363 +0000 UTC m=+642.890580435" Jul 07 18:55:48.341418 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:48.341387 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:48.341828 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:48.341465 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:48.353997 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:48.353979 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:55:48.807022 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:55:48.806942 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw" Jul 07 18:56:06.127258 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.127218 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r"] Jul 07 18:56:06.131032 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.131009 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.133462 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.133444 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 07 18:56:06.145299 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.145273 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r"] Jul 07 18:56:06.170989 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.170953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.170989 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.170988 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.171168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.171026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845df359-edcf-4c19-9440-921b15293f0c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.171168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.171063 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.171168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.171094 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc8kd\" (UniqueName: \"kubernetes.io/projected/845df359-edcf-4c19-9440-921b15293f0c-kube-api-access-sc8kd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.171168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.171121 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.271785 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.271753 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc8kd\" (UniqueName: \"kubernetes.io/projected/845df359-edcf-4c19-9440-921b15293f0c-kube-api-access-sc8kd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.271932 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.271879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.272013 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.271995 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.272137 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.272112 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.272220 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.272196 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845df359-edcf-4c19-9440-921b15293f0c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.272276 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.272228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.272346 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.272321 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.272404 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.272391 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.272520 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.272501 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.274780 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.274752 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/845df359-edcf-4c19-9440-921b15293f0c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.274983 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.274962 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845df359-edcf-4c19-9440-921b15293f0c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.282060 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.282040 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc8kd\" (UniqueName: \"kubernetes.io/projected/845df359-edcf-4c19-9440-921b15293f0c-kube-api-access-sc8kd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r\" (UID: \"845df359-edcf-4c19-9440-921b15293f0c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.446525 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.446425 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:06.588927 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.588898 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r"] Jul 07 18:56:06.590101 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:56:06.590079 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod845df359_edcf_4c19_9440_921b15293f0c.slice/crio-fab99a0c1528fe98576920d5ddfb6860024e0e68831138e6b6a9d05bde214a6a WatchSource:0}: Error finding container fab99a0c1528fe98576920d5ddfb6860024e0e68831138e6b6a9d05bde214a6a: Status 404 returned error can't find the container with id fab99a0c1528fe98576920d5ddfb6860024e0e68831138e6b6a9d05bde214a6a Jul 07 18:56:06.883163 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.883130 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" event={"ID":"845df359-edcf-4c19-9440-921b15293f0c","Type":"ContainerStarted","Data":"7c28b31eb7e2dc67e9ea493ad4c18b7b6f3841afc459a5d3f4c113c198a8d437"} Jul 07 18:56:06.883319 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.883177 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" event={"ID":"845df359-edcf-4c19-9440-921b15293f0c","Type":"ContainerStarted","Data":"fab99a0c1528fe98576920d5ddfb6860024e0e68831138e6b6a9d05bde214a6a"} Jul 07 18:56:06.910638 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:06.910591 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" podStartSLOduration=0.714961861 podStartE2EDuration="910.575367ms" totalImagesPullingTime="195.613506ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:06.907683827 +0000 UTC m=+668.932801899" watchObservedRunningTime="2026-07-07 18:56:06.910575367 +0000 UTC m=+668.935693440" Jul 07 18:56:16.446609 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:16.446573 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:16.447114 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:16.446627 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:16.459006 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:16.458979 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:16.944683 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:16.944638 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r" Jul 07 18:56:53.716630 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.716546 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76dc978b69-j2qhb"] Jul 07 18:56:53.720025 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.720005 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76dc978b69-j2qhb" Jul 07 18:56:53.723405 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.723384 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-v8gnw\"" Jul 07 18:56:53.723511 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.723459 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 18:56:53.731168 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.731146 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76dc978b69-j2qhb"] Jul 07 18:56:53.806385 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.806355 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7-tls-cert\") pod \"authorino-76dc978b69-j2qhb\" (UID: \"fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7\") " pod="kuadrant-system/authorino-76dc978b69-j2qhb" Jul 07 18:56:53.806557 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.806403 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgcfg\" (UniqueName: \"kubernetes.io/projected/fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7-kube-api-access-cgcfg\") pod \"authorino-76dc978b69-j2qhb\" (UID: \"fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7\") " pod="kuadrant-system/authorino-76dc978b69-j2qhb" Jul 07 18:56:53.907732 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.907699 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7-tls-cert\") pod \"authorino-76dc978b69-j2qhb\" (UID: \"fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7\") " pod="kuadrant-system/authorino-76dc978b69-j2qhb" Jul 07 18:56:53.907894 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.907757 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgcfg\" (UniqueName: \"kubernetes.io/projected/fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7-kube-api-access-cgcfg\") pod \"authorino-76dc978b69-j2qhb\" (UID: \"fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7\") " pod="kuadrant-system/authorino-76dc978b69-j2qhb" Jul 07 18:56:53.910258 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.910242 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7-tls-cert\") pod \"authorino-76dc978b69-j2qhb\" (UID: \"fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7\") " pod="kuadrant-system/authorino-76dc978b69-j2qhb" Jul 07 18:56:53.918122 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:53.918100 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgcfg\" (UniqueName: \"kubernetes.io/projected/fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7-kube-api-access-cgcfg\") pod \"authorino-76dc978b69-j2qhb\" (UID: \"fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7\") " pod="kuadrant-system/authorino-76dc978b69-j2qhb" Jul 07 18:56:54.030027 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:54.029939 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76dc978b69-j2qhb" Jul 07 18:56:54.181718 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:54.181341 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76dc978b69-j2qhb"] Jul 07 18:56:54.184163 ip-10-0-141-163 kubenswrapper[2582]: W0707 18:56:54.184138 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb8b03de_c1bd_4ff4_ad86_6a8d546a7fc7.slice/crio-dc8d55ece9123b45af56232e28886a424b2150224eeca55f60599059e27c4a94 WatchSource:0}: Error finding container dc8d55ece9123b45af56232e28886a424b2150224eeca55f60599059e27c4a94: Status 404 returned error can't find the container with id dc8d55ece9123b45af56232e28886a424b2150224eeca55f60599059e27c4a94 Jul 07 18:56:55.128336 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:55.128299 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76dc978b69-j2qhb" event={"ID":"fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7","Type":"ContainerStarted","Data":"f533c2517b53029b1a1b0ae6a0072049ad5a3f6c5e2f1d22c85aeffc725047ee"} Jul 07 18:56:55.128336 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:55.128341 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76dc978b69-j2qhb" event={"ID":"fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7","Type":"ContainerStarted","Data":"dc8d55ece9123b45af56232e28886a424b2150224eeca55f60599059e27c4a94"} Jul 07 18:56:55.158597 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:56:55.158535 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76dc978b69-j2qhb" podStartSLOduration=1.785235373 podStartE2EDuration="2.158518466s" totalImagesPullingTime="373.283093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:55.157777437 +0000 UTC m=+717.182895514" watchObservedRunningTime="2026-07-07 18:56:55.158518466 +0000 UTC m=+717.183636539" Jul 07 18:59:58.599526 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:59:58.599416 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:59:58.606174 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:59:58.606154 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 18:59:58.606578 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:59:58.606559 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 18:59:58.612737 ip-10-0-141-163 kubenswrapper[2582]: I0707 18:59:58.612720 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:04:58.632363 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:04:58.632257 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:04:58.639160 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:04:58.639139 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:04:58.639292 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:04:58.639165 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:04:58.645574 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:04:58.645556 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:09:58.663906 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:09:58.663788 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:09:58.670854 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:09:58.670826 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:09:58.672521 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:09:58.672503 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:09:58.678789 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:09:58.678772 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:14:58.701944 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:14:58.701822 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:14:58.708631 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:14:58.708503 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:14:58.712845 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:14:58.712827 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:14:58.719047 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:14:58.719031 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:19:58.733322 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:19:58.733208 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:19:58.740094 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:19:58.740076 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:19:58.746288 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:19:58.746268 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:19:58.756900 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:19:58.756884 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:23:11.661028 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.660995 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5"] Jul 07 19:23:11.664464 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.664446 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.667972 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.667946 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dedce5\"/\"default-dockercfg-6w8cd\"" Jul 07 19:23:11.667972 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.667970 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dedce5\"/\"kube-root-ca.crt\"" Jul 07 19:23:11.668143 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.667978 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dedce5\"/\"rate-test-model-dedce5-kserve-self-signed-certs\"" Jul 07 19:23:11.668143 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.667970 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dedce5\"/\"openshift-service-ca.crt\"" Jul 07 19:23:11.676209 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.676190 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5"] Jul 07 19:23:11.746595 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.746565 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-home\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.746595 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.746594 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tmp-dir\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.746811 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.746618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tls-certs\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.746811 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.746635 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdkmm\" (UniqueName: \"kubernetes.io/projected/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-kube-api-access-rdkmm\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.746811 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.746734 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-model-cache\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.746811 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.746755 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-dshm\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.847981 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.847932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-model-cache\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.847981 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.847986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-dshm\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.848230 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.848053 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-home\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.848230 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.848069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tmp-dir\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.848230 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.848100 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tls-certs\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.848230 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.848124 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdkmm\" (UniqueName: \"kubernetes.io/projected/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-kube-api-access-rdkmm\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.848614 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.848398 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-model-cache\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.848614 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.848456 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-home\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.848614 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.848508 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tmp-dir\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.850408 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.850390 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-dshm\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.850574 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.850557 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tls-certs\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.859800 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.859777 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdkmm\" (UniqueName: \"kubernetes.io/projected/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-kube-api-access-rdkmm\") pod \"rate-test-model-dedce5-kserve-66c46f9955-8klh5\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:11.973197 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:11.973099 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:12.113218 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:12.113195 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5"] Jul 07 19:23:12.115520 ip-10-0-141-163 kubenswrapper[2582]: W0707 19:23:12.115493 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1d29ce9_0bbe_4a97_a9f0_d0c1d1d62b2a.slice/crio-ba5eef6461bb3aa0730dbd09900f9bdaa6f4ba45004a76f3d57527376603fa3d WatchSource:0}: Error finding container ba5eef6461bb3aa0730dbd09900f9bdaa6f4ba45004a76f3d57527376603fa3d: Status 404 returned error can't find the container with id ba5eef6461bb3aa0730dbd09900f9bdaa6f4ba45004a76f3d57527376603fa3d Jul 07 19:23:12.139968 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:12.139928 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" event={"ID":"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a","Type":"ContainerStarted","Data":"ba5eef6461bb3aa0730dbd09900f9bdaa6f4ba45004a76f3d57527376603fa3d"} Jul 07 19:23:13.146608 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:13.146566 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" event={"ID":"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a","Type":"ContainerStarted","Data":"e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292"} Jul 07 19:23:13.167575 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:13.167526 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" podStartSLOduration=2.16750984 podStartE2EDuration="2.16750984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:23:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:23:13.165758608 +0000 UTC m=+2295.190876691" watchObservedRunningTime="2026-07-07 19:23:13.16750984 +0000 UTC m=+2295.192627912" Jul 07 19:23:21.973335 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:21.973295 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:21.973870 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:21.973358 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:21.985951 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:21.985930 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:23:22.204382 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:23:22.204352 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:24:25.557640 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:25.557606 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5"] Jul 07 19:24:40.578524 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.578478 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" podUID="f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" containerName="main" containerID="cri-o://e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292" gracePeriod=45 Jul 07 19:24:40.830709 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.830353 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:24:40.906150 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906118 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-dshm\") pod \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " Jul 07 19:24:40.906303 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906235 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tmp-dir\") pod \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " Jul 07 19:24:40.906303 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906271 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdkmm\" (UniqueName: \"kubernetes.io/projected/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-kube-api-access-rdkmm\") pod \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " Jul 07 19:24:40.906376 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906328 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-home\") pod \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " Jul 07 19:24:40.906376 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906362 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-model-cache\") pod \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " Jul 07 19:24:40.906507 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906477 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tls-certs\") pod \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\" (UID: \"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a\") " Jul 07 19:24:40.906565 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906529 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" (UID: "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:24:40.906622 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906573 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-model-cache" (OuterVolumeSpecName: "model-cache") pod "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" (UID: "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:24:40.906723 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906700 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-home" (OuterVolumeSpecName: "home") pod "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" (UID: "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:24:40.906883 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906866 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tmp-dir\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 19:24:40.906942 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906889 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-home\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 19:24:40.906942 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.906902 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-model-cache\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 19:24:40.908480 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.908456 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-dshm" (OuterVolumeSpecName: "dshm") pod "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" (UID: "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:24:40.908934 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.908901 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" (UID: "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:24:40.909102 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:40.908997 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-kube-api-access-rdkmm" (OuterVolumeSpecName: "kube-api-access-rdkmm") pod "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" (UID: "f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a"). InnerVolumeSpecName "kube-api-access-rdkmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:24:41.007880 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.007845 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdkmm\" (UniqueName: \"kubernetes.io/projected/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-kube-api-access-rdkmm\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 19:24:41.007880 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.007876 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-tls-certs\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 19:24:41.007880 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.007886 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a-dshm\") on node \"ip-10-0-141-163.ec2.internal\" DevicePath \"\"" Jul 07 19:24:41.593098 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.593065 2582 generic.go:356] "Generic (PLEG): container finished" podID="f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" containerID="e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292" exitCode=0 Jul 07 19:24:41.593498 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.593110 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" Jul 07 19:24:41.593498 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.593115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" event={"ID":"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a","Type":"ContainerDied","Data":"e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292"} Jul 07 19:24:41.593498 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.593154 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5" event={"ID":"f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a","Type":"ContainerDied","Data":"ba5eef6461bb3aa0730dbd09900f9bdaa6f4ba45004a76f3d57527376603fa3d"} Jul 07 19:24:41.593498 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.593176 2582 scope.go:117] "RemoveContainer" containerID="e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292" Jul 07 19:24:41.602617 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.602597 2582 scope.go:117] "RemoveContainer" containerID="e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292" Jul 07 19:24:41.602909 ip-10-0-141-163 kubenswrapper[2582]: E0707 19:24:41.602890 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292\": container with ID starting with e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292 not found: ID does not exist" containerID="e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292" Jul 07 19:24:41.602984 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.602917 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292"} err="failed to get container status \"e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292\": rpc error: code = NotFound desc = could not find container \"e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292\": container with ID starting with e1d3b37df835126c893352f2865a4abe8565b958435cb9c101311d1480d5a292 not found: ID does not exist" Jul 07 19:24:41.617094 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.617070 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5"] Jul 07 19:24:41.622026 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:41.622004 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-dedce5/rate-test-model-dedce5-kserve-66c46f9955-8klh5"] Jul 07 19:24:42.585483 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:42.585446 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" path="/var/lib/kubelet/pods/f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a/volumes" Jul 07 19:24:58.769677 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:58.769559 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:24:58.777128 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:58.777106 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:24:58.786774 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:58.786753 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:24:58.793738 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:24:58.793716 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7x6bl_cca766b7-da36-4ad6-95f3-3357c28ba6bf/ovn-acl-logging/0.log" Jul 07 19:25:20.320593 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:20.320514 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76dc978b69-j2qhb_fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7/authorino/0.log" Jul 07 19:25:25.935616 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:25.935590 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-545cccdbd8-6xl9x_2dfeb259-f3af-4a62-8bee-c03dd68ef2ee/copy-manifests/0.log" Jul 07 19:25:25.949098 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:25.949076 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-545cccdbd8-6xl9x_2dfeb259-f3af-4a62-8bee-c03dd68ef2ee/manager/0.log" Jul 07 19:25:27.608679 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:27.608631 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76dc978b69-j2qhb_fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7/authorino/0.log" Jul 07 19:25:27.849970 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:27.849941 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zwl8q_7362ab14-47ce-499f-9375-e8f1d48f82b2/manager/0.log" Jul 07 19:25:27.964590 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:27.964507 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wc2gc_86b4d633-a357-4fc0-8699-f24e76b2e387/kuadrant-console-plugin/0.log" Jul 07 19:25:28.455444 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:28.455415 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xthgk_b17f25e6-7e6f-4a70-8117-5139268e32b2/manager/0.log" Jul 07 19:25:28.805010 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:28.804932 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678xx2vk_3fe74e65-fce9-476c-9abd-0518cab2c81b/istio-proxy/0.log" Jul 07 19:25:29.043093 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:29.043065 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d58d67c5c-rgvhh_3d59c838-e730-4eed-aa2f-f8d6fe323a4e/kube-auth-proxy/0.log" Jul 07 19:25:29.520068 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:29.520037 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cvxvt_47ee66d0-9c2c-40c8-82f1-0509753fe20f/payload-processing/0.log" Jul 07 19:25:30.319909 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:30.319875 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5q6bw_3d6b6d38-93b2-460f-bbe7-db3cbd2c0089/main/0.log" Jul 07 19:25:30.548970 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:30.548945 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-75x9r_845df359-edcf-4c19-9440-921b15293f0c/main/0.log" Jul 07 19:25:38.387865 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:38.387838 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b4hlp_8af537b1-f55e-42ec-9269-1f55dc81a5ec/global-pull-secret-syncer/0.log" Jul 07 19:25:38.530738 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:38.530711 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lfcks_6b5c3ee7-abd0-4f1e-a55b-974c4614593a/konnectivity-agent/0.log" Jul 07 19:25:38.621833 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:38.621800 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-163.ec2.internal_5b50e609bf668c243e15e77d050b97b9/haproxy/0.log" Jul 07 19:25:43.506326 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:43.506289 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76dc978b69-j2qhb_fb8b03de-c1bd-4ff4-ad86-6a8d546a7fc7/authorino/0.log" Jul 07 19:25:43.584710 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:43.584678 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zwl8q_7362ab14-47ce-499f-9375-e8f1d48f82b2/manager/0.log" Jul 07 19:25:43.618801 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:43.618772 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wc2gc_86b4d633-a357-4fc0-8699-f24e76b2e387/kuadrant-console-plugin/0.log" Jul 07 19:25:43.966443 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:43.966413 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xthgk_b17f25e6-7e6f-4a70-8117-5139268e32b2/manager/0.log" Jul 07 19:25:45.968716 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:45.968614 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-rv5jt_012c2fba-0533-43e7-b7c3-7d3476290739/cluster-monitoring-operator/0.log" Jul 07 19:25:46.006682 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.006449 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-9nvxd_4bbfb7b9-88f7-434d-a586-e764e34d5a7b/kube-state-metrics/0.log" Jul 07 19:25:46.031480 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.031439 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-9nvxd_4bbfb7b9-88f7-434d-a586-e764e34d5a7b/kube-rbac-proxy-main/0.log" Jul 07 19:25:46.061035 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.061000 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-9nvxd_4bbfb7b9-88f7-434d-a586-e764e34d5a7b/kube-rbac-proxy-self/0.log" Jul 07 19:25:46.124275 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.124244 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-bjt8r_ab2c05e2-daf2-4807-863a-410c44a93324/monitoring-plugin/0.log" Jul 07 19:25:46.341372 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.341348 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m4ndg_896c42fc-177e-4e20-aa39-efcb2c534269/node-exporter/0.log" Jul 07 19:25:46.367883 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.367860 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m4ndg_896c42fc-177e-4e20-aa39-efcb2c534269/kube-rbac-proxy/0.log" Jul 07 19:25:46.397022 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.396997 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m4ndg_896c42fc-177e-4e20-aa39-efcb2c534269/init-textfile/0.log" Jul 07 19:25:46.432326 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.432291 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-6zkdt_209a5a3a-a456-41ae-b816-c430bcbb51ce/kube-rbac-proxy-main/0.log" Jul 07 19:25:46.464183 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.464160 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-6zkdt_209a5a3a-a456-41ae-b816-c430bcbb51ce/kube-rbac-proxy-self/0.log" Jul 07 19:25:46.489945 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.489919 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-6zkdt_209a5a3a-a456-41ae-b816-c430bcbb51ce/openshift-state-metrics/0.log" Jul 07 19:25:46.537738 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.537714 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e1f9e42-1173-43c2-8a1f-f1cbe42377b7/prometheus/0.log" Jul 07 19:25:46.560229 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.560201 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e1f9e42-1173-43c2-8a1f-f1cbe42377b7/config-reloader/0.log" Jul 07 19:25:46.587940 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.587917 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e1f9e42-1173-43c2-8a1f-f1cbe42377b7/thanos-sidecar/0.log" Jul 07 19:25:46.616623 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.616525 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e1f9e42-1173-43c2-8a1f-f1cbe42377b7/kube-rbac-proxy-web/0.log" Jul 07 19:25:46.645154 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.645128 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e1f9e42-1173-43c2-8a1f-f1cbe42377b7/kube-rbac-proxy/0.log" Jul 07 19:25:46.675077 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.675047 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e1f9e42-1173-43c2-8a1f-f1cbe42377b7/kube-rbac-proxy-thanos/0.log" Jul 07 19:25:46.711147 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.711122 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e1f9e42-1173-43c2-8a1f-f1cbe42377b7/init-config-reloader/0.log" Jul 07 19:25:46.806237 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.806209 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-rrvf9_e51a18a3-d221-4a7c-aa76-3126282eb043/prometheus-operator-admission-webhook/0.log" Jul 07 19:25:46.853290 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.853265 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c6c54cfcc-dwlvt_9c7240e5-fcd1-4cd8-9a4f-16c6f476b624/telemeter-client/0.log" Jul 07 19:25:46.878877 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.878805 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c6c54cfcc-dwlvt_9c7240e5-fcd1-4cd8-9a4f-16c6f476b624/reload/0.log" Jul 07 19:25:46.906112 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.906084 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c6c54cfcc-dwlvt_9c7240e5-fcd1-4cd8-9a4f-16c6f476b624/kube-rbac-proxy/0.log" Jul 07 19:25:46.948393 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.948364 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6db7c474f4-cdsjf_80771db7-4da9-494a-9f3b-0233ef6003c2/thanos-query/0.log" Jul 07 19:25:46.993401 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:46.993371 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6db7c474f4-cdsjf_80771db7-4da9-494a-9f3b-0233ef6003c2/kube-rbac-proxy-web/0.log" Jul 07 19:25:47.023770 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:47.023740 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6db7c474f4-cdsjf_80771db7-4da9-494a-9f3b-0233ef6003c2/kube-rbac-proxy/0.log" Jul 07 19:25:47.057939 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:47.057911 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6db7c474f4-cdsjf_80771db7-4da9-494a-9f3b-0233ef6003c2/prom-label-proxy/0.log" Jul 07 19:25:47.086935 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:47.086908 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6db7c474f4-cdsjf_80771db7-4da9-494a-9f3b-0233ef6003c2/kube-rbac-proxy-rules/0.log" Jul 07 19:25:47.123076 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:47.123047 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6db7c474f4-cdsjf_80771db7-4da9-494a-9f3b-0233ef6003c2/kube-rbac-proxy-metrics/0.log" Jul 07 19:25:48.024030 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.023999 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58"] Jul 07 19:25:48.024407 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.024397 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" containerName="main" Jul 07 19:25:48.024448 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.024409 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" containerName="main" Jul 07 19:25:48.024515 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.024504 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1d29ce9-0bbe-4a97-a9f0-d0c1d1d62b2a" containerName="main" Jul 07 19:25:48.027922 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.027902 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.030160 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.030141 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sls4v\"/\"kube-root-ca.crt\"" Jul 07 19:25:48.030992 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.030969 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sls4v\"/\"openshift-service-ca.crt\"" Jul 07 19:25:48.031071 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.030971 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sls4v\"/\"default-dockercfg-phqzb\"" Jul 07 19:25:48.038026 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.037998 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58"] Jul 07 19:25:48.153533 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.153492 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-lib-modules\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.153728 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.153551 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-proc\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.153728 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.153619 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjs2c\" (UniqueName: \"kubernetes.io/projected/e157312f-7ed3-42f3-87c8-334004768c9c-kube-api-access-gjs2c\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.153728 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.153720 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-sys\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.153830 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.153790 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-podres\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.254993 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.254961 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-sys\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.255160 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.255043 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-podres\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.255160 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.255068 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-lib-modules\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.255160 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.255081 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-sys\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.255160 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.255107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-proc\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.255160 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.255136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjs2c\" (UniqueName: \"kubernetes.io/projected/e157312f-7ed3-42f3-87c8-334004768c9c-kube-api-access-gjs2c\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.255388 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.255186 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-lib-modules\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.255388 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.255214 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-podres\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.255388 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.255237 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e157312f-7ed3-42f3-87c8-334004768c9c-proc\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.264933 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.264911 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjs2c\" (UniqueName: \"kubernetes.io/projected/e157312f-7ed3-42f3-87c8-334004768c9c-kube-api-access-gjs2c\") pod \"perf-node-gather-daemonset-qms58\" (UID: \"e157312f-7ed3-42f3-87c8-334004768c9c\") " pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.338566 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.338538 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.369472 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.369446 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-wbqxk_acc883c3-9957-47d7-adef-481a0399c18a/networking-console-plugin/0.log" Jul 07 19:25:48.489798 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.489767 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58"] Jul 07 19:25:48.491796 ip-10-0-141-163 kubenswrapper[2582]: W0707 19:25:48.491768 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode157312f_7ed3_42f3_87c8_334004768c9c.slice/crio-13e64f258c3cfd4ff212e1ad2b155a40882b9668e5a3ce978fabf13a7b44510f WatchSource:0}: Error finding container 13e64f258c3cfd4ff212e1ad2b155a40882b9668e5a3ce978fabf13a7b44510f: Status 404 returned error can't find the container with id 13e64f258c3cfd4ff212e1ad2b155a40882b9668e5a3ce978fabf13a7b44510f Jul 07 19:25:48.928283 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.928252 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" event={"ID":"e157312f-7ed3-42f3-87c8-334004768c9c","Type":"ContainerStarted","Data":"c37894f184c0de9333849fa4130867882cb7655a26231d67c1a981be86b004de"} Jul 07 19:25:48.928473 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.928309 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" event={"ID":"e157312f-7ed3-42f3-87c8-334004768c9c","Type":"ContainerStarted","Data":"13e64f258c3cfd4ff212e1ad2b155a40882b9668e5a3ce978fabf13a7b44510f"} Jul 07 19:25:48.928473 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.928373 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:48.948063 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:48.948015 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" podStartSLOduration=0.947999105 podStartE2EDuration="947.999105ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:25:48.945901366 +0000 UTC m=+2450.971019439" watchObservedRunningTime="2026-07-07 19:25:48.947999105 +0000 UTC m=+2450.973117178" Jul 07 19:25:49.053075 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:49.053048 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/2.log" Jul 07 19:25:49.057269 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:49.057242 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wwpsl_8383d520-8b64-45f6-b79d-a50b06f0e20c/console-operator/3.log" Jul 07 19:25:49.611727 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:49.611698 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-p9njm_2396709f-c39c-4aa3-9900-80758e35764e/download-server/0.log" Jul 07 19:25:50.193762 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:50.193732 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-t4cwq_33da68e1-9493-4e0d-acf2-8bcad8072fa5/volume-data-source-validator/0.log" Jul 07 19:25:51.388605 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:51.388576 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-snt25_b9576f40-fc27-45a0-a17d-d19d7da6676b/dns/0.log" Jul 07 19:25:51.418028 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:51.417978 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-snt25_b9576f40-fc27-45a0-a17d-d19d7da6676b/kube-rbac-proxy/0.log" Jul 07 19:25:51.459695 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:51.459634 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5wwvt_039885f1-e2ef-4c36-8fe1-c1b0a1a023ea/dns-node-resolver/0.log" Jul 07 19:25:52.120609 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:52.120575 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8lmc_0d3043a3-d10f-4fef-bbe1-7f9d18666671/node-ca/0.log" Jul 07 19:25:53.212870 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:53.212839 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678xx2vk_3fe74e65-fce9-476c-9abd-0518cab2c81b/istio-proxy/0.log" Jul 07 19:25:53.350395 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:53.350367 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d58d67c5c-rgvhh_3d59c838-e730-4eed-aa2f-f8d6fe323a4e/kube-auth-proxy/0.log" Jul 07 19:25:53.529839 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:53.529750 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cvxvt_47ee66d0-9c2c-40c8-82f1-0509753fe20f/payload-processing/0.log" Jul 07 19:25:54.292939 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:54.292906 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2wzjs_45d31a00-8411-45f4-832d-12ddf08417f5/serve-healthcheck-canary/0.log" Jul 07 19:25:54.943840 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:54.943813 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sls4v/perf-node-gather-daemonset-qms58" Jul 07 19:25:55.005610 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:55.005575 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cw9rd_095c0673-2f70-4197-98fa-194a67fb7715/kube-rbac-proxy/0.log" Jul 07 19:25:55.031038 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:55.031013 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cw9rd_095c0673-2f70-4197-98fa-194a67fb7715/exporter/0.log" Jul 07 19:25:55.059317 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:55.059286 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cw9rd_095c0673-2f70-4197-98fa-194a67fb7715/extractor/0.log" Jul 07 19:25:58.146946 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:58.146910 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-545cccdbd8-6xl9x_2dfeb259-f3af-4a62-8bee-c03dd68ef2ee/manager/0.log" Jul 07 19:25:58.166992 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:58.166968 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-545cccdbd8-6xl9x_2dfeb259-f3af-4a62-8bee-c03dd68ef2ee/copy-manifests/0.log" Jul 07 19:25:58.280262 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:58.280233 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678xx2vk_3fe74e65-fce9-476c-9abd-0518cab2c81b/istio-proxy/0.log" Jul 07 19:25:58.388183 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:58.388108 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d58d67c5c-rgvhh_3d59c838-e730-4eed-aa2f-f8d6fe323a4e/kube-auth-proxy/0.log" Jul 07 19:25:58.511966 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:58.511930 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cvxvt_47ee66d0-9c2c-40c8-82f1-0509753fe20f/payload-processing/0.log" Jul 07 19:25:59.781750 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:59.781728 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66b96648c9-h4wgv_c9d1da0e-66c0-422d-935e-ac6a2f7af12e/manager/0.log" Jul 07 19:25:59.825986 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:59.825959 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678xx2vk_3fe74e65-fce9-476c-9abd-0518cab2c81b/istio-proxy/0.log" Jul 07 19:25:59.938895 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:25:59.938869 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d58d67c5c-rgvhh_3d59c838-e730-4eed-aa2f-f8d6fe323a4e/kube-auth-proxy/0.log" Jul 07 19:26:00.070415 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:26:00.070370 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cvxvt_47ee66d0-9c2c-40c8-82f1-0509753fe20f/payload-processing/0.log" Jul 07 19:26:04.840051 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:26:04.840022 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-7vwsh_275646cf-6982-42dc-838f-5bb27fb8ab45/migrator/0.log" Jul 07 19:26:04.869940 ip-10-0-141-163 kubenswrapper[2582]: I0707 19:26:04.869909 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-7vwsh_275646cf-6982-42dc-838f-5bb27fb8ab45/graceful-termination/0.log"