Jul 01 18:35:14.777135 ip-10-0-134-114 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 01 18:35:14.777148 ip-10-0-134-114 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 01 18:35:14.777158 ip-10-0-134-114 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 01 18:35:14.777385 ip-10-0-134-114 systemd[1]: Failed to start Kubernetes Kubelet. Jul 01 18:35:25.987542 ip-10-0-134-114 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 01 18:35:25.987560 ip-10-0-134-114 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f45fa1eadfbc4d73b5c1c5835eb6be8b -- Jul 01 18:37:03.361135 ip-10-0-134-114 systemd[1]: Starting Kubernetes Kubelet... Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.775735 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783945 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783959 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783963 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783966 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783969 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783975 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783978 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:03.814541 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783981 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783985 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783988 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783994 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.783998 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784002 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784005 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784008 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784011 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784014 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784018 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784023 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784026 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784035 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784038 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784041 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784044 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784047 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784050 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:03.816016 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784053 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784056 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784059 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784062 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784066 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784069 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784072 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784075 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784078 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784081 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784084 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784087 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784090 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784094 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784098 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784102 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784105 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784108 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784112 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:03.816952 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784114 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784117 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784120 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784123 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784125 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784128 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784132 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784134 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784138 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784141 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784144 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784146 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784149 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784152 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784155 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784159 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784163 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784166 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784169 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784172 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:03.817898 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784175 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784179 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784183 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784186 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784189 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784192 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784197 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784200 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784203 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784206 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784209 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784212 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784214 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784217 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784220 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784224 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784227 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784229 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784232 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784235 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:03.818848 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784238 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784241 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784245 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784248 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784251 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784254 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784257 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784260 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784263 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784266 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784269 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784272 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784274 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.784278 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786792 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786802 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786805 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786809 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786812 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786822 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:03.819792 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786826 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786829 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786832 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786837 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786841 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786844 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786848 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786851 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786854 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786858 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786861 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786865 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786868 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786871 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786874 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786877 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786880 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786883 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786887 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:03.820724 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786890 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786893 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786896 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786899 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786901 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786904 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786907 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786911 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786928 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786932 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786935 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786938 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786941 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786946 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786948 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786951 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786954 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786957 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786960 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:03.821694 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786963 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786965 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786969 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786971 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786974 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786977 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786980 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786983 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786986 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786989 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786992 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786995 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.786998 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787002 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787005 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787007 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787010 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787013 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787015 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787018 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787021 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:03.822662 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787024 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787026 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787029 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787032 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787034 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787037 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787040 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787043 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787047 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787050 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787053 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787056 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787058 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787061 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787064 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787066 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787069 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787072 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787075 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787078 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:03.823631 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787085 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787088 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787091 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787093 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787096 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787099 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787102 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787105 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787107 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787112 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787116 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787119 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787122 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787125 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787572 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787578 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787581 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787583 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787590 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:03.826854 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787593 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787596 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787599 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787602 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787604 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787607 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787610 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787613 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787616 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787619 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787622 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787625 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787627 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787632 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787634 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787637 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787640 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787642 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787645 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787648 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:03.827828 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787650 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787654 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787656 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787659 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787662 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787665 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787668 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787670 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787673 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787676 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787678 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787683 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787686 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787689 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787692 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787694 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787697 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787700 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787702 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787707 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:03.829443 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787711 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787714 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787717 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787720 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787723 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787726 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787729 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787732 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787735 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787738 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787741 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787743 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787746 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787749 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787752 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787755 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787757 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787760 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787763 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:03.830385 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787766 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787771 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787774 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787777 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787781 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787784 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787793 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787796 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787798 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787802 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787804 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787807 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787810 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787812 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787815 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787818 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787820 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787823 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787827 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787830 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:03.831574 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787832 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787835 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787838 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787841 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787843 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787847 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787851 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787854 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787857 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787859 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787862 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787865 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787867 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787870 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.787873 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.787964 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.787974 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.787981 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.787985 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.787996 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788000 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 18:37:03.832515 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788004 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788015 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788018 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788021 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788025 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788028 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788032 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788035 2562 flags.go:64] FLAG: --cgroup-root="" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788038 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788042 2562 flags.go:64] FLAG: --client-ca-file="" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788045 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788048 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788054 2562 flags.go:64] FLAG: --cluster-domain="" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788057 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788060 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788064 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788067 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788071 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788075 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788078 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788081 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788084 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788088 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788091 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 18:37:03.837653 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788094 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788097 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788101 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788106 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788109 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788112 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788115 2562 flags.go:64] FLAG: --enable-server="true" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788119 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788124 2562 flags.go:64] FLAG: --event-burst="100" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788127 2562 flags.go:64] FLAG: --event-qps="50" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788130 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788133 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788136 2562 flags.go:64] FLAG: --eviction-hard="" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788140 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788143 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788146 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788149 2562 flags.go:64] FLAG: --eviction-soft="" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788154 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788157 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788160 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788163 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788166 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788169 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788172 2562 flags.go:64] FLAG: --feature-gates="" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788176 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 18:37:03.839287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788180 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788183 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788186 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788190 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788193 2562 flags.go:64] FLAG: --help="false" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788196 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-134-114.ec2.internal" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788200 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788203 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788206 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788210 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788215 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788218 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788221 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788224 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788227 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788230 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788233 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788236 2562 flags.go:64] FLAG: --kube-reserved="" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788239 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788242 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788245 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788248 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788251 2562 flags.go:64] FLAG: --lock-file="" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788254 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 18:37:03.840149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788259 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788262 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788268 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788271 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788274 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788277 2562 flags.go:64] FLAG: --logging-format="text" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788280 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788283 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788287 2562 flags.go:64] FLAG: --manifest-url="" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788290 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788294 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788297 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788302 2562 flags.go:64] FLAG: --max-pods="110" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788305 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788308 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788311 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788314 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788317 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788321 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788325 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788332 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788335 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788338 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 18:37:03.840853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788341 2562 flags.go:64] FLAG: --pod-cidr="" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788345 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788351 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788354 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788358 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788361 2562 flags.go:64] FLAG: --port="10250" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788364 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788367 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0de03ab8a3dd5783c" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788370 2562 flags.go:64] FLAG: --qos-reserved="" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788373 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788377 2562 flags.go:64] FLAG: --register-node="true" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788380 2562 flags.go:64] FLAG: --register-with-taints="" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788385 2562 flags.go:64] FLAG: --registry-burst="10" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788388 2562 flags.go:64] FLAG: --registry-qps="5" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788390 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788393 2562 flags.go:64] FLAG: --reserved-memory="" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788398 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788401 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788404 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788407 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788410 2562 flags.go:64] FLAG: --runonce="false" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788413 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788416 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788419 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788422 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 18:37:03.841651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788425 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788429 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788433 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788437 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788440 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788443 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788445 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788448 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788451 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788455 2562 flags.go:64] FLAG: --system-cgroups="" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788458 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788463 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788467 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788469 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788474 2562 flags.go:64] FLAG: --tls-min-version="" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788477 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788480 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788484 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788488 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788491 2562 flags.go:64] FLAG: --v="2" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788495 2562 flags.go:64] FLAG: --version="false" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788499 2562 flags.go:64] FLAG: --vmodule="" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788503 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.788506 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788666 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:04.018698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788671 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:03.861408 ip-10-0-134-114 systemd[1]: Started Kubernetes Kubelet. Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788676 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788680 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788684 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788687 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788690 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788693 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788696 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788699 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788703 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788706 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788708 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788711 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788714 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788717 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788720 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788722 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788731 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788734 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788736 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:04.046225 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788739 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788742 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788745 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788747 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788751 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788754 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788757 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788760 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788762 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788765 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788768 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788771 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788774 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788777 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788780 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788782 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788785 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788788 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788791 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788793 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:04.047844 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788796 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788800 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788803 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788805 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788808 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788810 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788813 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788816 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788819 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788821 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788830 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788833 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788835 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788838 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788848 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788852 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788855 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788858 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788861 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788864 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:04.050650 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788866 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788871 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788875 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788878 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788882 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788885 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788888 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788890 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788893 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788896 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788898 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788901 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788904 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788908 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788911 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788926 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788930 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788932 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788935 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788938 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:04.052135 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788941 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788945 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788949 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788958 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788961 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788964 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788967 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788970 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788974 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788977 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788980 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788982 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788985 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788988 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788990 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788993 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788996 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:04.053572 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.788998 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.789004 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.800273 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.800294 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.800314 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.800320 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800391 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800398 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800401 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800405 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800408 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800411 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800414 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800416 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:04.056741 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800419 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800424 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800429 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800433 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800437 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800440 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800444 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800447 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800452 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800455 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800458 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800462 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800464 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800467 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800470 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800473 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800476 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800479 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800482 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:04.057698 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800485 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800488 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800493 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800496 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800499 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800502 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800505 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800508 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800511 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800514 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800517 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800520 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800523 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800526 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800528 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800531 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800534 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800538 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800540 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:04.059950 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800543 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800546 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800550 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800553 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800556 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800558 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800561 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800564 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800566 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800570 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800574 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800578 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800581 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800584 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800588 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800591 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800595 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800598 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800601 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:04.061107 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800604 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800606 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800609 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800612 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800614 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800617 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800620 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800623 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800626 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800628 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800632 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800635 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800638 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800640 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800644 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800648 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800651 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800654 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800656 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800659 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:04.063652 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800662 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800665 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800668 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800670 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800673 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800675 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800678 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800681 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800684 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800686 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800690 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800692 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800695 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800698 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.800703 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:37:04.065116 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800827 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800832 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800837 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800839 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800842 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800847 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800850 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800854 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800858 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800861 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800864 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800867 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800872 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800874 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800877 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800880 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800883 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800886 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800888 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:04.066464 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800891 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800894 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800896 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800900 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800904 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800907 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800910 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800913 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800934 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800937 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800940 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800943 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800946 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800949 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800952 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800955 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800958 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800961 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800963 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800966 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:04.067509 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800969 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800973 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800976 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800979 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800982 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800986 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800989 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800992 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800995 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.800998 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801000 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801003 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801006 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801009 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801012 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801014 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801017 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801020 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801023 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801025 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:04.068490 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801028 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801031 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801035 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801037 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801040 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801043 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801046 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801049 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801051 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801054 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801057 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801059 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801062 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801066 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801069 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801072 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801074 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801079 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801081 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801084 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801087 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:04.069496 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801090 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801093 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801096 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801099 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801101 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801104 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801107 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801110 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801113 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801115 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801118 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801121 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801123 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801126 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801129 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801132 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801135 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801138 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:04.070523 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:03.801141 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.801146 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.801264 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.805215 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.806591 2562 server.go:1019] "Starting client certificate rotation" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.806701 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.806738 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.809308 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.812741 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.826906 2562 log.go:25] "Validated CRI v1 runtime API" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.829462 2562 log.go:25] "Validated CRI v1 image API" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.830322 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 18:37:04.071528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.833127 2562 fs.go:135] Filesystem UUIDs: map[7152c387-5b54-4115-8451-cf1c31e10304:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 caf908ec-18f8-49a4-a2e5-fd2801fcccb3:/dev/nvme0n1p3] Jul 01 18:37:04.072485 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.833145 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 18:37:04.072944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.838815 2562 manager.go:216] Machine: {Timestamp:2026-07-01 18:37:03.837232481 +0000 UTC m=+0.352823714 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101079 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2419aa878ae6e324fd7b7774c72457 SystemUUID:ec2419aa-878a-e6e3-24fd-7b7774c72457 BootID:f45fa1ea-dfbc-4d73-b5c1-c5835eb6be8b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e9:62:05:5e:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e9:62:05:5e:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:e8:75:86:d8:55 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 18:37:04.072944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.838943 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 18:37:04.072944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.839063 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 18:37:04.072944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.842339 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 18:37:04.072944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.842365 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-114.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 18:37:04.072944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.843191 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 18:37:04.072944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.843202 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 18:37:04.072944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.843209 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.844686 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.845583 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.848591 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.848722 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.849407 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.849420 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.849437 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.849447 2562 kubelet.go:398] "Adding apiserver pod source" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.849456 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.849791 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.849806 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.852325 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.854462 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.854496 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855222 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855236 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855244 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855250 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855259 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855265 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855352 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855360 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855366 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855373 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855382 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.855391 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.856062 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.856069 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.857825 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.860244 2562 server.go:1268] "Started kubelet" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.860310 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.860319 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.860359 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.861566 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.863641 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-114.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.863667 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.863669 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.863812 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 01 18:37:04.073443 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.867194 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.868720 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.869584 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.869888 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870219 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870226 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870389 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870441 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870454 2562 factory.go:55] Registering systemd factory Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870463 2562 factory.go:223] Registration of the systemd container factory successfully Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870483 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870493 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870884 2562 factory.go:153] Registering CRI-O factory Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870902 2562 factory.go:223] Registration of the crio container factory successfully Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870927 2562 factory.go:103] Registering Raw factory Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.870940 2562 manager.go:1196] Started watching for new ooms in manager Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.871386 2562 manager.go:349] Starting recovery of all containers Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.872366 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.872458 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.872218 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-114.ec2.internal.18be3e51e03623bd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-114.ec2.internal,UID:ip-10-0-134-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-114.ec2.internal,},FirstTimestamp:2026-07-01 18:37:03.860212669 +0000 UTC m=+0.375803902,LastTimestamp:2026-07-01 18:37:03.860212669 +0000 UTC m=+0.375803902,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-114.ec2.internal,}" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.882441 2562 manager.go:354] Recovery completed Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.887128 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.889152 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.889197 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.889213 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.889754 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.889763 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.889766 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.889800 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.890537 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.890565 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.890575 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.891000 2562 policy_none.go:49] "None policy: Start" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.891014 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 18:37:04.075471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.891025 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.892215 2562 policy_none.go:47] "Start" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.900618 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-114.ec2.internal.18be3e51e1f02a50 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-114.ec2.internal,UID:ip-10-0-134-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-114.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-114.ec2.internal,},FirstTimestamp:2026-07-01 18:37:03.889181264 +0000 UTC m=+0.404772501,LastTimestamp:2026-07-01 18:37:03.889181264 +0000 UTC m=+0.404772501,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-114.ec2.internal,}" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.902454 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-114.ec2.internal.18be3e51e1f08410 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-114.ec2.internal,UID:ip-10-0-134-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-114.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-114.ec2.internal,},FirstTimestamp:2026-07-01 18:37:03.88920424 +0000 UTC m=+0.404795475,LastTimestamp:2026-07-01 18:37:03.88920424 +0000 UTC m=+0.404795475,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-114.ec2.internal,}" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.912747 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nhngb" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.912774 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-114.ec2.internal.18be3e51e1f0c23b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-114.ec2.internal,UID:ip-10-0-134-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-114.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-114.ec2.internal,},FirstTimestamp:2026-07-01 18:37:03.889220155 +0000 UTC m=+0.404811393,LastTimestamp:2026-07-01 18:37:03.889220155 +0000 UTC m=+0.404811393,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-114.ec2.internal,}" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.922010 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nhngb" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.927335 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-114.ec2.internal.18be3e51e1f02a50\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-114.ec2.internal.18be3e51e1f02a50 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-114.ec2.internal,UID:ip-10-0-134-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-114.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-114.ec2.internal,},FirstTimestamp:2026-07-01 18:37:03.889181264 +0000 UTC m=+0.404772501,LastTimestamp:2026-07-01 18:37:03.890551785 +0000 UTC m=+0.406143017,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-114.ec2.internal,}" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.933671 2562 manager.go:337] "Starting Device Plugin manager" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.933702 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.933712 2562 server.go:87] "Starting device plugin registration server" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.934005 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 18:37:04.078716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.934018 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.934120 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.934229 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.934240 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.934812 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.934881 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.952688 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.955431 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.955466 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.955487 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:03.955520 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:03.957863 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.034490 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.036627 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.036660 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.036674 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.036698 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.047302 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.047326 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-114.ec2.internal\": node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.056025 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal"] Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.056099 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.057725 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.057755 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.057766 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059026 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059177 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059779 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059799 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059802 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059810 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059822 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.059834 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.061108 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.061133 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:04.079693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.061900 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:04.083405 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.061951 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:04.083405 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.061972 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:04.083405 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.072012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a15d2a32d4de4fde9197b362accbd8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal\" (UID: \"f5a15d2a32d4de4fde9197b362accbd8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.083405 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.072038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2d9920d8a67dc75b8a30a40184be48bc-config\") pod \"kube-apiserver-proxy-ip-10-0-134-114.ec2.internal\" (UID: \"2d9920d8a67dc75b8a30a40184be48bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.083405 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.072056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5a15d2a32d4de4fde9197b362accbd8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal\" (UID: \"f5a15d2a32d4de4fde9197b362accbd8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.083405 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.076457 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.178202 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.172899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5a15d2a32d4de4fde9197b362accbd8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal\" (UID: \"f5a15d2a32d4de4fde9197b362accbd8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.178202 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.172935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a15d2a32d4de4fde9197b362accbd8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal\" (UID: \"f5a15d2a32d4de4fde9197b362accbd8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.178202 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.172953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2d9920d8a67dc75b8a30a40184be48bc-config\") pod \"kube-apiserver-proxy-ip-10-0-134-114.ec2.internal\" (UID: \"2d9920d8a67dc75b8a30a40184be48bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.178202 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.172996 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2d9920d8a67dc75b8a30a40184be48bc-config\") pod \"kube-apiserver-proxy-ip-10-0-134-114.ec2.internal\" (UID: \"2d9920d8a67dc75b8a30a40184be48bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.178202 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.172996 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5a15d2a32d4de4fde9197b362accbd8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal\" (UID: \"f5a15d2a32d4de4fde9197b362accbd8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.178202 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.173072 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a15d2a32d4de4fde9197b362accbd8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal\" (UID: \"f5a15d2a32d4de4fde9197b362accbd8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:04.178202 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.177016 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.290536 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.278109 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.407532 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.378957 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.525157 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.479703 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.646411 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.580304 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.763035 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.681026 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.870415 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.781654 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.870415 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.806961 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 18:37:04.870415 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.807158 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 18:37:04.870415 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.807171 2562 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a6aca39276a0c4baf9ab75240026f0ea-2b11aa60b6fc00a6.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-134-114.ec2.internal.18be3e51e1f02a50\": read tcp 10.0.134.114:35902->100.60.142.175:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-134-114.ec2.internal.18be3e51e1f02a50 default 5561 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-114.ec2.internal,UID:ip-10-0-134-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-114.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-114.ec2.internal,},FirstTimestamp:2026-07-01 18:37:03 +0000 UTC,LastTimestamp:2026-07-01 18:37:04.059813699 +0000 UTC m=+0.575404934,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-114.ec2.internal,}" Jul 01 18:37:04.975435 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.882705 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.975435 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.926968 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 18:32:04 +0000 UTC" deadline="2028-04-04 11:02:36.585745694 +0000 UTC" Jul 01 18:37:04.975435 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.926997 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15424h25m31.658751191s" Jul 01 18:37:04.982929 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:04.982896 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:04.983975 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.983958 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:04.985112 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.985093 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:04.985165 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.985129 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:04.985165 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:04.985144 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:05.029994 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.029966 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-114.ec2.internal\" not found" node="ip-10-0-134-114.ec2.internal" Jul 01 18:37:05.031978 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.031961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:05.035140 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.035120 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-114.ec2.internal\" not found" node="ip-10-0-134-114.ec2.internal" Jul 01 18:37:05.038603 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.038582 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" Jul 01 18:37:05.083260 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.083223 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:05.141618 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.141593 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:05.184116 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.184088 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:05.284788 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.284703 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-114.ec2.internal\" not found" Jul 01 18:37:05.292017 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.291999 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:05.341767 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.341733 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:05.370768 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.370737 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" Jul 01 18:37:05.386303 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.386279 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 18:37:05.387502 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.387489 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" Jul 01 18:37:05.398159 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.398142 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 18:37:05.564022 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:05.563987 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9920d8a67dc75b8a30a40184be48bc.slice/crio-2fa805dd883054200feb8b17da1a6ff6a0cbf3ee445c1185ee84e101a4cac796 WatchSource:0}: Error finding container 2fa805dd883054200feb8b17da1a6ff6a0cbf3ee445c1185ee84e101a4cac796: Status 404 returned error can't find the container with id 2fa805dd883054200feb8b17da1a6ff6a0cbf3ee445c1185ee84e101a4cac796 Jul 01 18:37:05.564395 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:05.564371 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a15d2a32d4de4fde9197b362accbd8.slice/crio-f6c065abcfef608c9d775c0110cca0d3ba64f06260031d303ed71a21aa4ce8ea WatchSource:0}: Error finding container f6c065abcfef608c9d775c0110cca0d3ba64f06260031d303ed71a21aa4ce8ea: Status 404 returned error can't find the container with id f6c065abcfef608c9d775c0110cca0d3ba64f06260031d303ed71a21aa4ce8ea Jul 01 18:37:05.569958 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.569938 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 18:37:05.641302 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.641268 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:05.854566 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.854532 2562 apiserver.go:52] "Watching apiserver" Jul 01 18:37:05.860116 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.860092 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 18:37:05.860488 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.860465 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-d9nnh","kube-system/global-pull-secret-syncer-j8lms","kube-system/konnectivity-agent-l8k74","openshift-dns/node-resolver-fcb9x","openshift-image-registry/node-ca-sh9jq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal","openshift-multus/multus-6srdv","openshift-multus/multus-additional-cni-plugins-t9995","openshift-multus/network-metrics-daemon-npbg7","kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t","openshift-cluster-node-tuning-operator/tuned-b4hlx","openshift-network-diagnostics/network-check-target-67lhc","openshift-network-operator/iptables-alerter-fsvrm"] Jul 01 18:37:05.862964 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.862951 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.864229 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.864207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:05.864342 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.864285 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:05.865544 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.865523 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 18:37:05.865709 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.865645 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 18:37:05.865709 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.865656 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 18:37:05.865826 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.865710 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 18:37:05.866005 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.865989 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8sjzs\"" Jul 01 18:37:05.866055 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.866041 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.867696 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.867677 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.868098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.868075 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hwttj\"" Jul 01 18:37:05.868245 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.868123 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 18:37:05.868245 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.868142 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 18:37:05.868980 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.868963 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.869644 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.869626 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2k6qn\"" Jul 01 18:37:05.869804 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.869783 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 18:37:05.869910 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.869871 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 18:37:05.870259 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.870239 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.870648 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.870626 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 18:37:05.871425 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.871410 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 18:37:05.871512 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.871452 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 18:37:05.871654 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.871638 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:05.871732 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.871713 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:05.871912 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.871885 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xf8qd\"" Jul 01 18:37:05.872012 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.871935 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 18:37:05.873309 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.872742 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:37:05.873309 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.873084 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bj5lc\"" Jul 01 18:37:05.873309 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.873197 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 18:37:05.876302 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.875950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.877711 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.877692 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:05.878733 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.878714 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 18:37:05.878829 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.878744 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 18:37:05.879256 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.879195 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:05.879331 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.879260 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:05.879979 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.879958 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 18:37:05.880075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880025 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 18:37:05.880075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880048 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 18:37:05.880170 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880053 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 18:37:05.880212 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880171 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-knxvr\"" Jul 01 18:37:05.880401 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880384 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 18:37:05.880401 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880398 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 18:37:05.880590 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880576 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lzgn6\"" Jul 01 18:37:05.880734 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880720 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 18:37:05.880808 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.880783 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:05.881896 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.881876 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-lib-modules\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.882019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.881901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f250d29-e5fb-4c99-8662-29dddf0e37c5-tmp-dir\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.882019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.881957 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.882019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.881991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-tmp\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.882179 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-cni-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.882179 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd36d396-98d4-4666-9a09-c575ead013da-cni-binary-copy\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.882179 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-system-cni-dir\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.882179 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882094 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-cnibin\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.882179 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-cni-binary-copy\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.882179 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882148 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.882179 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882167 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-host\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.882179 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882176 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd96d\" (UniqueName: \"kubernetes.io/projected/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-kube-api-access-kd96d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-host\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882314 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-tuned\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-os-release\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882389 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-socket-dir-parent\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-k8s-cni-cncf-io\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-netns\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-cni-bin\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd36d396-98d4-4666-9a09-c575ead013da-multus-daemon-config\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-os-release\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.882540 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882570 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysctl-conf\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvtn6\" (UniqueName: \"kubernetes.io/projected/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-kube-api-access-xvtn6\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54q5q\" (UniqueName: \"kubernetes.io/projected/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-kube-api-access-54q5q\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-sys\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-var-lib-kubelet\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f250d29-e5fb-4c99-8662-29dddf0e37c5-hosts-file\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg5n7\" (UniqueName: \"kubernetes.io/projected/2f250d29-e5fb-4c99-8662-29dddf0e37c5-kube-api-access-tg5n7\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-multus-certs\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-etc-kubernetes\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysconfig\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882837 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-kubernetes\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882860 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-run\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-system-cni-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-cnibin\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-cni-multus\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.882991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-kubelet\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.883017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-hostroot\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.883043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-serviceca\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.883069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-conf-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.883115 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpvvg\" (UniqueName: \"kubernetes.io/projected/dd36d396-98d4-4666-9a09-c575ead013da-kube-api-access-dpvvg\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.883179 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk7q9\" (UniqueName: \"kubernetes.io/projected/315b7908-1f18-4b3f-800d-05cf10e7da1e-kube-api-access-wk7q9\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.883216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-modprobe-d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.883243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysctl-d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.883557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.883273 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-systemd\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.884256 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.884236 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:37:05.884344 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.884256 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cn5xm\"" Jul 01 18:37:05.884426 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.884409 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kxmk9\"" Jul 01 18:37:05.884480 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.884416 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 18:37:05.884564 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.884549 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 18:37:05.884605 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.884552 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 18:37:05.884818 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.884803 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 18:37:05.885703 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.885683 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 18:37:05.910995 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.910974 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tqmkt" Jul 01 18:37:05.920560 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.920527 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tqmkt" Jul 01 18:37:05.960395 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.960339 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" event={"ID":"f5a15d2a32d4de4fde9197b362accbd8","Type":"ContainerStarted","Data":"f6c065abcfef608c9d775c0110cca0d3ba64f06260031d303ed71a21aa4ce8ea"} Jul 01 18:37:05.961631 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.961602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" event={"ID":"2d9920d8a67dc75b8a30a40184be48bc","Type":"ContainerStarted","Data":"2fa805dd883054200feb8b17da1a6ff6a0cbf3ee445c1185ee84e101a4cac796"} Jul 01 18:37:05.971541 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.971514 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 18:37:05.983887 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.983864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysctl-conf\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.984013 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.983897 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvtn6\" (UniqueName: \"kubernetes.io/projected/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-kube-api-access-xvtn6\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.984013 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.983916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54q5q\" (UniqueName: \"kubernetes.io/projected/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-kube-api-access-54q5q\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:05.984013 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.983942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/264efc66-6510-4a19-94f2-908be89caa25-ovn-node-metrics-cert\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.984175 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-sys\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.984175 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-var-lib-kubelet\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.984270 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984169 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysctl-conf\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.984270 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984186 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg5n7\" (UniqueName: \"kubernetes.io/projected/2f250d29-e5fb-4c99-8662-29dddf0e37c5-kube-api-access-tg5n7\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.984270 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984208 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-sys\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.984270 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984238 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-var-lib-kubelet\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.984438 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984330 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.984438 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-kubernetes\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.984438 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-cnibin\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984438 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-cni-multus\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984438 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984425 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-kubernetes\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-kubelet\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984485 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-cnibin\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-conf-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984515 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-conf-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984520 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5932d17-9b81-4e7e-94ce-1db0cded279d-kubelet-config\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984519 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-cni-multus\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984556 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0aeff29-79df-4f33-88db-7a200c306c69-iptables-alerter-script\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984562 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-kubelet\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984579 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0aeff29-79df-4f33-88db-7a200c306c69-host-slash\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpvvg\" (UniqueName: \"kubernetes.io/projected/dd36d396-98d4-4666-9a09-c575ead013da-kube-api-access-dpvvg\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.984651 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-env-overrides\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-modprobe-d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysctl-d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984897 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-systemd\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-cni-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984927 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-modprobe-d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd36d396-98d4-4666-9a09-c575ead013da-cni-binary-copy\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984967 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-etc-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984989 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysctl-d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984971 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-systemd\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.984995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-node-log\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-cni-netd\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.985072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swhj5\" (UniqueName: \"kubernetes.io/projected/3d184d95-2b81-483d-b7fd-7879bbc2de5d-kube-api-access-swhj5\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d36698d3-80dd-4298-92ff-30f664e015fc-agent-certs\") pod \"konnectivity-agent-l8k74\" (UID: \"d36698d3-80dd-4298-92ff-30f664e015fc\") " pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985110 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-cni-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985127 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-system-cni-dir\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-cnibin\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985159 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-host\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd96d\" (UniqueName: \"kubernetes.io/projected/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-kube-api-access-kd96d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985182 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-system-cni-dir\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985209 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-cnibin\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985215 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-socket-dir-parent\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985243 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-host\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985261 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-multus-socket-dir-parent\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985300 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-run-ovn-kubernetes\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.985661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985317 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-device-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985335 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-tuned\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-netns\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d36698d3-80dd-4298-92ff-30f664e015fc-konnectivity-ca\") pod \"konnectivity-agent-l8k74\" (UID: \"d36698d3-80dd-4298-92ff-30f664e015fc\") " pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-os-release\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985443 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985467 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-netns\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-cni-bin\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985561 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f250d29-e5fb-4c99-8662-29dddf0e37c5-hosts-file\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985572 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-os-release\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-multus-certs\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-etc-kubernetes\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.986260 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985626 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-multus-certs\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985614 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f250d29-e5fb-4c99-8662-29dddf0e37c5-hosts-file\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985633 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd36d396-98d4-4666-9a09-c575ead013da-cni-binary-copy\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/315b7908-1f18-4b3f-800d-05cf10e7da1e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-systemd\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985670 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-etc-kubernetes\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysconfig\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-run\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985750 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-system-cni-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985763 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-sysconfig\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985768 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-run\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985804 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-hostroot\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985820 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-slash\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985836 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-run-netns\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.987537 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985852 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-ovnkube-config\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985880 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-registration-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985897 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-hostroot\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-serviceca\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985895 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-system-cni-dir\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.985942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-kubelet\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-var-lib-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgkdf\" (UniqueName: \"kubernetes.io/projected/c0aeff29-79df-4f33-88db-7a200c306c69-kube-api-access-rgkdf\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986127 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk7q9\" (UniqueName: \"kubernetes.io/projected/315b7908-1f18-4b3f-800d-05cf10e7da1e-kube-api-access-wk7q9\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986151 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-lib-modules\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f250d29-e5fb-4c99-8662-29dddf0e37c5-tmp-dir\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986195 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-systemd-units\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986215 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-ovn\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-ovnkube-script-lib\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.988031 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986257 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2w7w\" (UniqueName: \"kubernetes.io/projected/264efc66-6510-4a19-94f2-908be89caa25-kube-api-access-n2w7w\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986300 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-lib-modules\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986309 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-serviceca\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-tmp\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986398 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-log-socket\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986443 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-cni-binary-copy\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f250d29-e5fb-4c99-8662-29dddf0e37c5-tmp-dir\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-host\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-os-release\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-k8s-cni-cncf-io\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd36d396-98d4-4666-9a09-c575ead013da-multus-daemon-config\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-socket-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986739 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-run-k8s-cni-cncf-io\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.986743 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986790 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-cni-bin\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-host\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986826 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-os-release\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986825 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986749 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd36d396-98d4-4666-9a09-c575ead013da-host-var-lib-cni-bin\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.986942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-sys-fs\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.986962 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs podName:d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:06.486939519 +0000 UTC m=+3.002530739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs") pod "network-metrics-daemon-npbg7" (UID: "d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.987010 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5932d17-9b81-4e7e-94ce-1db0cded279d-dbus\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.987047 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.987040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:05.988857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.987334 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd36d396-98d4-4666-9a09-c575ead013da-multus-daemon-config\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:05.989290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.987539 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/315b7908-1f18-4b3f-800d-05cf10e7da1e-cni-binary-copy\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:05.989290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.989255 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-etc-tuned\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.989351 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:05.989284 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-tmp\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:05.999280 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.999123 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:05.999280 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.999149 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:05.999280 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.999162 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2kmh8 for pod openshift-network-diagnostics/network-check-target-67lhc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:05.999280 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:05.999231 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8 podName:b40e8345-4927-47e8-8cd9-dba01f13a362 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:06.499212754 +0000 UTC m=+3.014803998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2kmh8" (UniqueName: "kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8") pod "network-check-target-67lhc" (UID: "b40e8345-4927-47e8-8cd9-dba01f13a362") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:06.000336 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.000309 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg5n7\" (UniqueName: \"kubernetes.io/projected/2f250d29-e5fb-4c99-8662-29dddf0e37c5-kube-api-access-tg5n7\") pod \"node-resolver-fcb9x\" (UID: \"2f250d29-e5fb-4c99-8662-29dddf0e37c5\") " pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:06.000432 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.000309 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpvvg\" (UniqueName: \"kubernetes.io/projected/dd36d396-98d4-4666-9a09-c575ead013da-kube-api-access-dpvvg\") pod \"multus-6srdv\" (UID: \"dd36d396-98d4-4666-9a09-c575ead013da\") " pod="openshift-multus/multus-6srdv" Jul 01 18:37:06.000432 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.000384 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk7q9\" (UniqueName: \"kubernetes.io/projected/315b7908-1f18-4b3f-800d-05cf10e7da1e-kube-api-access-wk7q9\") pod \"multus-additional-cni-plugins-t9995\" (UID: \"315b7908-1f18-4b3f-800d-05cf10e7da1e\") " pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:06.000553 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.000532 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd96d\" (UniqueName: \"kubernetes.io/projected/8c6f8a24-db08-4a4b-8577-6c9d2f14277b-kube-api-access-kd96d\") pod \"tuned-b4hlx\" (UID: \"8c6f8a24-db08-4a4b-8577-6c9d2f14277b\") " pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:06.001000 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.000978 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvtn6\" (UniqueName: \"kubernetes.io/projected/e4c2815f-c0b3-4524-b6ce-96d3c5aba855-kube-api-access-xvtn6\") pod \"node-ca-sh9jq\" (UID: \"e4c2815f-c0b3-4524-b6ce-96d3c5aba855\") " pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:06.001098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.001079 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54q5q\" (UniqueName: \"kubernetes.io/projected/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-kube-api-access-54q5q\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5932d17-9b81-4e7e-94ce-1db0cded279d-kubelet-config\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0aeff29-79df-4f33-88db-7a200c306c69-iptables-alerter-script\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087481 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0aeff29-79df-4f33-88db-7a200c306c69-host-slash\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-env-overrides\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087538 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-etc-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-node-log\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-cni-netd\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swhj5\" (UniqueName: \"kubernetes.io/projected/3d184d95-2b81-483d-b7fd-7879bbc2de5d-kube-api-access-swhj5\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d36698d3-80dd-4298-92ff-30f664e015fc-agent-certs\") pod \"konnectivity-agent-l8k74\" (UID: \"d36698d3-80dd-4298-92ff-30f664e015fc\") " pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.087690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087696 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-run-ovn-kubernetes\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087697 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0aeff29-79df-4f33-88db-7a200c306c69-host-slash\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087710 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5932d17-9b81-4e7e-94ce-1db0cded279d-kubelet-config\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087722 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-device-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087772 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087851 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-etc-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087913 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.087991 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-cni-netd\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088042 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-node-log\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088048 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-run-ovn-kubernetes\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d36698d3-80dd-4298-92ff-30f664e015fc-konnectivity-ca\") pod \"konnectivity-agent-l8k74\" (UID: \"d36698d3-80dd-4298-92ff-30f664e015fc\") " pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-device-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-cni-bin\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.088464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-systemd\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-slash\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088265 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-run-netns\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088270 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-env-overrides\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088295 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-ovnkube-config\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088297 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088343 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-slash\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088327 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-registration-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088396 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-run-netns\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088425 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0aeff29-79df-4f33-88db-7a200c306c69-iptables-alerter-script\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088454 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-registration-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088477 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-cni-bin\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088480 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-kubelet\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-systemd\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088573 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-var-lib-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgkdf\" (UniqueName: \"kubernetes.io/projected/c0aeff29-79df-4f33-88db-7a200c306c69-kube-api-access-rgkdf\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088680 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-var-lib-openvswitch\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088722 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-systemd-units\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-ovn\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088797 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-host-kubelet\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088880 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d36698d3-80dd-4298-92ff-30f664e015fc-konnectivity-ca\") pod \"konnectivity-agent-l8k74\" (UID: \"d36698d3-80dd-4298-92ff-30f664e015fc\") " pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088896 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-run-ovn\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088982 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-systemd-units\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-ovnkube-config\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.088800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-ovnkube-script-lib\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2w7w\" (UniqueName: \"kubernetes.io/projected/264efc66-6510-4a19-94f2-908be89caa25-kube-api-access-n2w7w\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089379 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-log-socket\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089382 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/264efc66-6510-4a19-94f2-908be89caa25-ovnkube-script-lib\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.089769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-socket-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089478 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-sys-fs\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5932d17-9b81-4e7e-94ce-1db0cded279d-dbus\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/264efc66-6510-4a19-94f2-908be89caa25-ovn-node-metrics-cert\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089667 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-socket-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089702 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/264efc66-6510-4a19-94f2-908be89caa25-log-socket\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089768 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5932d17-9b81-4e7e-94ce-1db0cded279d-dbus\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.089799 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d184d95-2b81-483d-b7fd-7879bbc2de5d-sys-fs\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.089828 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:06.090611 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.089930 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret podName:a5932d17-9b81-4e7e-94ce-1db0cded279d nodeName:}" failed. No retries permitted until 2026-07-01 18:37:06.589909201 +0000 UTC m=+3.105500422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret") pod "global-pull-secret-syncer-j8lms" (UID: "a5932d17-9b81-4e7e-94ce-1db0cded279d") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:06.092030 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.091555 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d36698d3-80dd-4298-92ff-30f664e015fc-agent-certs\") pod \"konnectivity-agent-l8k74\" (UID: \"d36698d3-80dd-4298-92ff-30f664e015fc\") " pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:06.092697 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.092669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/264efc66-6510-4a19-94f2-908be89caa25-ovn-node-metrics-cert\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.098773 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.098739 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swhj5\" (UniqueName: \"kubernetes.io/projected/3d184d95-2b81-483d-b7fd-7879bbc2de5d-kube-api-access-swhj5\") pod \"aws-ebs-csi-driver-node-c792t\" (UID: \"3d184d95-2b81-483d-b7fd-7879bbc2de5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.098957 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.098936 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2w7w\" (UniqueName: \"kubernetes.io/projected/264efc66-6510-4a19-94f2-908be89caa25-kube-api-access-n2w7w\") pod \"ovnkube-node-d9nnh\" (UID: \"264efc66-6510-4a19-94f2-908be89caa25\") " pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.100244 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.100222 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgkdf\" (UniqueName: \"kubernetes.io/projected/c0aeff29-79df-4f33-88db-7a200c306c69-kube-api-access-rgkdf\") pod \"iptables-alerter-fsvrm\" (UID: \"c0aeff29-79df-4f33-88db-7a200c306c69\") " pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:06.177030 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.176923 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6srdv" Jul 01 18:37:06.182200 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.182175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t9995" Jul 01 18:37:06.188944 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.188913 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd36d396_98d4_4666_9a09_c575ead013da.slice/crio-cab363b01f5d9623f34fd7e2207cf12258a696182b48e1327b682b89dc6604b2 WatchSource:0}: Error finding container cab363b01f5d9623f34fd7e2207cf12258a696182b48e1327b682b89dc6604b2: Status 404 returned error can't find the container with id cab363b01f5d9623f34fd7e2207cf12258a696182b48e1327b682b89dc6604b2 Jul 01 18:37:06.190198 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.190104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fcb9x" Jul 01 18:37:06.194032 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.193908 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sh9jq" Jul 01 18:37:06.200140 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.200117 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" Jul 01 18:37:06.204616 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.204583 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f250d29_e5fb_4c99_8662_29dddf0e37c5.slice/crio-d6b90aad2811063fdbe89c55095c0a41068ae48fcf13601fcf02de8117496b4f WatchSource:0}: Error finding container d6b90aad2811063fdbe89c55095c0a41068ae48fcf13601fcf02de8117496b4f: Status 404 returned error can't find the container with id d6b90aad2811063fdbe89c55095c0a41068ae48fcf13601fcf02de8117496b4f Jul 01 18:37:06.204903 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.204881 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315b7908_1f18_4b3f_800d_05cf10e7da1e.slice/crio-8b41fda2f03ff33f5395dd5d00c9ae0666330b2b97dd7e5ad668c42fea23aef0 WatchSource:0}: Error finding container 8b41fda2f03ff33f5395dd5d00c9ae0666330b2b97dd7e5ad668c42fea23aef0: Status 404 returned error can't find the container with id 8b41fda2f03ff33f5395dd5d00c9ae0666330b2b97dd7e5ad668c42fea23aef0 Jul 01 18:37:06.206132 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.206112 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:06.210025 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.209990 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c2815f_c0b3_4524_b6ce_96d3c5aba855.slice/crio-f4fffc4ce05d83d204394da959bea9599436a03d21d845091358e439d8d0ec40 WatchSource:0}: Error finding container f4fffc4ce05d83d204394da959bea9599436a03d21d845091358e439d8d0ec40: Status 404 returned error can't find the container with id f4fffc4ce05d83d204394da959bea9599436a03d21d845091358e439d8d0ec40 Jul 01 18:37:06.211998 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.211776 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" Jul 01 18:37:06.217401 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.217230 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:06.221825 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.221793 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264efc66_6510_4a19_94f2_908be89caa25.slice/crio-26cfb5cc887cc331b075737d0c5f4abf44463cc8483465d8e3ae0103d6c6f95d WatchSource:0}: Error finding container 26cfb5cc887cc331b075737d0c5f4abf44463cc8483465d8e3ae0103d6c6f95d: Status 404 returned error can't find the container with id 26cfb5cc887cc331b075737d0c5f4abf44463cc8483465d8e3ae0103d6c6f95d Jul 01 18:37:06.223064 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.223039 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fsvrm" Jul 01 18:37:06.226417 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.226394 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6f8a24_db08_4a4b_8577_6c9d2f14277b.slice/crio-5b5800aca7b51084988c41e07578f77179efdd2365b0d60b1031be2a5b46a425 WatchSource:0}: Error finding container 5b5800aca7b51084988c41e07578f77179efdd2365b0d60b1031be2a5b46a425: Status 404 returned error can't find the container with id 5b5800aca7b51084988c41e07578f77179efdd2365b0d60b1031be2a5b46a425 Jul 01 18:37:06.236159 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.236131 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0aeff29_79df_4f33_88db_7a200c306c69.slice/crio-c959b6ac273d5b56aee557658939facacf9a8edb6dee8f02c71c03a01138197a WatchSource:0}: Error finding container c959b6ac273d5b56aee557658939facacf9a8edb6dee8f02c71c03a01138197a: Status 404 returned error can't find the container with id c959b6ac273d5b56aee557658939facacf9a8edb6dee8f02c71c03a01138197a Jul 01 18:37:06.237597 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.237574 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36698d3_80dd_4298_92ff_30f664e015fc.slice/crio-287e5069e5e15c882741fcabda2e97f7f02f47e64922dea60d89e0fbb26fa378 WatchSource:0}: Error finding container 287e5069e5e15c882741fcabda2e97f7f02f47e64922dea60d89e0fbb26fa378: Status 404 returned error can't find the container with id 287e5069e5e15c882741fcabda2e97f7f02f47e64922dea60d89e0fbb26fa378 Jul 01 18:37:06.238401 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:06.238380 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d184d95_2b81_483d_b7fd_7879bbc2de5d.slice/crio-54309c048e2c665c5810b5d76cec86e80afe2746ca9b94eddbb41a2b748c750c WatchSource:0}: Error finding container 54309c048e2c665c5810b5d76cec86e80afe2746ca9b94eddbb41a2b748c750c: Status 404 returned error can't find the container with id 54309c048e2c665c5810b5d76cec86e80afe2746ca9b94eddbb41a2b748c750c Jul 01 18:37:06.491894 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.491795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:06.492064 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.491968 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:06.492064 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.492044 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs podName:d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:07.492022129 +0000 UTC m=+4.007613371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs") pod "network-metrics-daemon-npbg7" (UID: "d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:06.594119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.593239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:06.594119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.593338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:06.594119 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.593490 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:06.594119 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.593561 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret podName:a5932d17-9b81-4e7e-94ce-1db0cded279d nodeName:}" failed. No retries permitted until 2026-07-01 18:37:07.593541186 +0000 UTC m=+4.109132413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret") pod "global-pull-secret-syncer-j8lms" (UID: "a5932d17-9b81-4e7e-94ce-1db0cded279d") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:06.594119 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.593656 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:06.594119 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.593675 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:06.594119 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.593687 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2kmh8 for pod openshift-network-diagnostics/network-check-target-67lhc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:06.594119 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.593738 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8 podName:b40e8345-4927-47e8-8cd9-dba01f13a362 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:07.593724865 +0000 UTC m=+4.109316088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kmh8" (UniqueName: "kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8") pod "network-check-target-67lhc" (UID: "b40e8345-4927-47e8-8cd9-dba01f13a362") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:06.923717 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.923582 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 18:32:06 +0000 UTC" deadline="2028-02-09 02:37:35.588551795 +0000 UTC" Jul 01 18:37:06.923717 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.923643 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14096h0m28.664912915s" Jul 01 18:37:06.956820 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.956270 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:06.956820 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:06.956420 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:06.967870 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.967764 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fcb9x" event={"ID":"2f250d29-e5fb-4c99-8662-29dddf0e37c5","Type":"ContainerStarted","Data":"d6b90aad2811063fdbe89c55095c0a41068ae48fcf13601fcf02de8117496b4f"} Jul 01 18:37:06.971081 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.971043 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fsvrm" event={"ID":"c0aeff29-79df-4f33-88db-7a200c306c69","Type":"ContainerStarted","Data":"c959b6ac273d5b56aee557658939facacf9a8edb6dee8f02c71c03a01138197a"} Jul 01 18:37:06.973080 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.973051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" event={"ID":"3d184d95-2b81-483d-b7fd-7879bbc2de5d","Type":"ContainerStarted","Data":"54309c048e2c665c5810b5d76cec86e80afe2746ca9b94eddbb41a2b748c750c"} Jul 01 18:37:06.977720 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.977620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"26cfb5cc887cc331b075737d0c5f4abf44463cc8483465d8e3ae0103d6c6f95d"} Jul 01 18:37:06.979984 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.979923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerStarted","Data":"8b41fda2f03ff33f5395dd5d00c9ae0666330b2b97dd7e5ad668c42fea23aef0"} Jul 01 18:37:06.981732 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.981622 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6srdv" event={"ID":"dd36d396-98d4-4666-9a09-c575ead013da","Type":"ContainerStarted","Data":"cab363b01f5d9623f34fd7e2207cf12258a696182b48e1327b682b89dc6604b2"} Jul 01 18:37:06.984236 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.984211 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l8k74" event={"ID":"d36698d3-80dd-4298-92ff-30f664e015fc","Type":"ContainerStarted","Data":"287e5069e5e15c882741fcabda2e97f7f02f47e64922dea60d89e0fbb26fa378"} Jul 01 18:37:06.987354 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.987326 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" event={"ID":"8c6f8a24-db08-4a4b-8577-6c9d2f14277b","Type":"ContainerStarted","Data":"5b5800aca7b51084988c41e07578f77179efdd2365b0d60b1031be2a5b46a425"} Jul 01 18:37:06.989580 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:06.989525 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sh9jq" event={"ID":"e4c2815f-c0b3-4524-b6ce-96d3c5aba855","Type":"ContainerStarted","Data":"f4fffc4ce05d83d204394da959bea9599436a03d21d845091358e439d8d0ec40"} Jul 01 18:37:07.503455 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:07.503416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:07.503640 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.503590 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:07.503700 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.503661 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs podName:d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:09.503640597 +0000 UTC m=+6.019231833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs") pod "network-metrics-daemon-npbg7" (UID: "d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:07.604133 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:07.604093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:07.604298 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:07.604187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:07.604377 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.604339 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:07.604440 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.604409 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret podName:a5932d17-9b81-4e7e-94ce-1db0cded279d nodeName:}" failed. No retries permitted until 2026-07-01 18:37:09.60438844 +0000 UTC m=+6.119979665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret") pod "global-pull-secret-syncer-j8lms" (UID: "a5932d17-9b81-4e7e-94ce-1db0cded279d") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:07.605137 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.604833 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:07.605137 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.604856 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:07.605137 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.604871 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2kmh8 for pod openshift-network-diagnostics/network-check-target-67lhc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:07.605137 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.604937 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8 podName:b40e8345-4927-47e8-8cd9-dba01f13a362 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:09.604903531 +0000 UTC m=+6.120494768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kmh8" (UniqueName: "kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8") pod "network-check-target-67lhc" (UID: "b40e8345-4927-47e8-8cd9-dba01f13a362") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:07.924168 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:07.924122 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 18:32:06 +0000 UTC" deadline="2028-03-03 15:47:13.321700787 +0000 UTC" Jul 01 18:37:07.924168 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:07.924167 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14661h10m5.397538909s" Jul 01 18:37:07.957067 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:07.956701 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:07.957067 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.956844 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:07.957746 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:07.957383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:07.957746 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:07.957519 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:08.956316 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:08.956278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:08.956863 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:08.956456 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:09.521192 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:09.521155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:09.521420 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.521371 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:09.521484 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.521439 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs podName:d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:13.521419497 +0000 UTC m=+10.037010753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs") pod "network-metrics-daemon-npbg7" (UID: "d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:09.622388 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:09.622350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:09.622566 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:09.622420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:09.622707 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.622581 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:09.622707 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.622600 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:09.622707 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.622614 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2kmh8 for pod openshift-network-diagnostics/network-check-target-67lhc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:09.622707 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.622668 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:09.622707 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.622682 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8 podName:b40e8345-4927-47e8-8cd9-dba01f13a362 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:13.622661802 +0000 UTC m=+10.138253026 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kmh8" (UniqueName: "kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8") pod "network-check-target-67lhc" (UID: "b40e8345-4927-47e8-8cd9-dba01f13a362") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:09.623020 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.622738 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret podName:a5932d17-9b81-4e7e-94ce-1db0cded279d nodeName:}" failed. No retries permitted until 2026-07-01 18:37:13.622719217 +0000 UTC m=+10.138310451 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret") pod "global-pull-secret-syncer-j8lms" (UID: "a5932d17-9b81-4e7e-94ce-1db0cded279d") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:09.955925 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:09.955888 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:09.956134 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.956055 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:09.956216 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:09.956147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:09.956286 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:09.956262 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:10.955970 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:10.955898 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:10.956510 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:10.956037 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:11.957170 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:11.956881 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:11.957170 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:11.956881 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:11.957170 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:11.957023 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:11.957170 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:11.957098 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:12.956358 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:12.956322 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:12.956541 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:12.956436 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:13.555767 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:13.555669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:13.556212 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.555875 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:13.556212 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.555957 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs podName:d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:21.555935478 +0000 UTC m=+18.071526709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs") pod "network-metrics-daemon-npbg7" (UID: "d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:13.656992 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:13.656948 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:13.657186 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:13.657034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:13.657186 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.657174 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:13.657298 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.657249 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret podName:a5932d17-9b81-4e7e-94ce-1db0cded279d nodeName:}" failed. No retries permitted until 2026-07-01 18:37:21.65723098 +0000 UTC m=+18.172822202 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret") pod "global-pull-secret-syncer-j8lms" (UID: "a5932d17-9b81-4e7e-94ce-1db0cded279d") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:13.657675 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.657651 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:13.657762 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.657685 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:13.657762 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.657698 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2kmh8 for pod openshift-network-diagnostics/network-check-target-67lhc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:13.657762 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.657743 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8 podName:b40e8345-4927-47e8-8cd9-dba01f13a362 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:21.657728589 +0000 UTC m=+18.173319810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kmh8" (UniqueName: "kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8") pod "network-check-target-67lhc" (UID: "b40e8345-4927-47e8-8cd9-dba01f13a362") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:13.960301 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:13.960268 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:13.960471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:13.960273 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:13.960471 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.960398 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:13.960587 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:13.960515 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:14.956490 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:14.955961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:14.956490 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:14.956095 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:15.955939 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:15.955909 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:15.955939 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:15.955938 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:15.956189 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:15.956035 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:15.956189 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:15.956114 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:16.955810 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:16.955775 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:16.956308 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:16.955905 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:17.956004 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:17.955963 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:17.956004 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:17.955984 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:17.956474 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:17.956102 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:17.956474 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:17.956225 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:18.956063 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:18.956025 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:18.956528 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:18.956153 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:19.956203 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:19.956165 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:19.956670 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:19.956304 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:19.956670 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:19.956363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:19.956670 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:19.956472 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:20.956129 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:20.956093 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:20.956353 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:20.956225 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:21.619773 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:21.619721 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:21.619974 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.619900 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:21.620035 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.619977 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs podName:d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:37.619955227 +0000 UTC m=+34.135546446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs") pod "network-metrics-daemon-npbg7" (UID: "d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:21.720299 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:21.720260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:21.720466 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:21.720326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:21.720466 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.720420 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:21.720466 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.720452 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:21.720611 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.720470 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:21.720611 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.720483 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2kmh8 for pod openshift-network-diagnostics/network-check-target-67lhc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:21.720611 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.720495 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret podName:a5932d17-9b81-4e7e-94ce-1db0cded279d nodeName:}" failed. No retries permitted until 2026-07-01 18:37:37.720479418 +0000 UTC m=+34.236070635 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret") pod "global-pull-secret-syncer-j8lms" (UID: "a5932d17-9b81-4e7e-94ce-1db0cded279d") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:21.720611 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.720533 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8 podName:b40e8345-4927-47e8-8cd9-dba01f13a362 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:37.720518517 +0000 UTC m=+34.236109775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kmh8" (UniqueName: "kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8") pod "network-check-target-67lhc" (UID: "b40e8345-4927-47e8-8cd9-dba01f13a362") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:21.956064 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:21.956038 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:21.956201 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.956143 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:21.956250 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:21.956216 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:21.956319 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:21.956302 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:22.956742 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:22.956483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:22.957234 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:22.956866 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:23.956450 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:23.956415 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:23.956622 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:23.956525 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:23.956622 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:23.956546 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:23.956722 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:23.956688 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:24.955996 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:24.955963 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:24.956421 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:24.956081 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:25.956650 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:25.956616 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:25.957068 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:25.956616 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:25.957068 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:25.956724 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:25.957068 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:25.956813 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:26.955919 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:26.955678 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:26.956079 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:26.955958 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:27.039232 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.039191 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" event={"ID":"2d9920d8a67dc75b8a30a40184be48bc","Type":"ContainerStarted","Data":"f9904db835d61bf72526d0d5a4b84c2cbaf6ed5784039a45b7a1f3a1f330afb4"} Jul 01 18:37:27.040657 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.040637 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" event={"ID":"3d184d95-2b81-483d-b7fd-7879bbc2de5d","Type":"ContainerStarted","Data":"13502eb5067356a4cc7fc108a4ae237a1ae98dd676d8bea7d972419241000a2b"} Jul 01 18:37:27.043335 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.043313 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:37:27.043658 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.043638 2562 generic.go:356] "Generic (PLEG): container finished" podID="264efc66-6510-4a19-94f2-908be89caa25" containerID="39ebfe1191eb853d87de23010c6fdb005dca4e3d671e48a2d6b49c4b9c9fa8f2" exitCode=1 Jul 01 18:37:27.043731 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.043711 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"be81370946ffeb20a1761b313812d8fefb1c77b87dff946cd6ad3a79e111c353"} Jul 01 18:37:27.043772 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.043761 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"44e7495b1aaf51317c558f82cdeaf8ff987982f79a5dddb97d67d2279ba4c0b6"} Jul 01 18:37:27.043816 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.043802 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"4a6d1054b9f44dfbba9347a3cc6026f014e91a60b37030884ecdaa07647c015e"} Jul 01 18:37:27.043886 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.043856 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"77fc756720bd69812c43abca80cf7ce89f5b255e1c78af61ee573cd3b89dfd3b"} Jul 01 18:37:27.043928 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.043896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerDied","Data":"39ebfe1191eb853d87de23010c6fdb005dca4e3d671e48a2d6b49c4b9c9fa8f2"} Jul 01 18:37:27.043974 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.043929 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"8ae4fe136984401373505beedf0e061edd4ee290b97a19cc7550f5c32f79b86e"} Jul 01 18:37:27.045151 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.045130 2562 generic.go:356] "Generic (PLEG): container finished" podID="315b7908-1f18-4b3f-800d-05cf10e7da1e" containerID="dc8bd97f0588547fdf28393d50c6c3ba08fb0d35387acfe206044138cd43cd59" exitCode=0 Jul 01 18:37:27.045234 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.045197 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerDied","Data":"dc8bd97f0588547fdf28393d50c6c3ba08fb0d35387acfe206044138cd43cd59"} Jul 01 18:37:27.046639 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.046616 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6srdv" event={"ID":"dd36d396-98d4-4666-9a09-c575ead013da","Type":"ContainerStarted","Data":"af2a5f2aa460a938faa9bd0ae89643a78cbccfe5ad1e3cee976da1f8836ec8b2"} Jul 01 18:37:27.048075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.048053 2562 generic.go:356] "Generic (PLEG): container finished" podID="f5a15d2a32d4de4fde9197b362accbd8" containerID="f5fa100cb2d0dc1381c6b49d15acb53a87da892d5e0a5a8bb442f562bf04d37b" exitCode=0 Jul 01 18:37:27.048160 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.048116 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" event={"ID":"f5a15d2a32d4de4fde9197b362accbd8","Type":"ContainerDied","Data":"f5fa100cb2d0dc1381c6b49d15acb53a87da892d5e0a5a8bb442f562bf04d37b"} Jul 01 18:37:27.049528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.049505 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l8k74" event={"ID":"d36698d3-80dd-4298-92ff-30f664e015fc","Type":"ContainerStarted","Data":"fad8e754a52d837edaf72c4f580c90021aadc48b7781a96ec33709b0ea1525f3"} Jul 01 18:37:27.050963 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.050945 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" event={"ID":"8c6f8a24-db08-4a4b-8577-6c9d2f14277b","Type":"ContainerStarted","Data":"6a33df1e6455055ab5e4087ffa1926c40dd813434d879c2943d6c34bf13fab78"} Jul 01 18:37:27.052452 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.052428 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sh9jq" event={"ID":"e4c2815f-c0b3-4524-b6ce-96d3c5aba855","Type":"ContainerStarted","Data":"d97f2e97e5d7a572b44000da3cca4d772c9fc1e506958280ff153710e9128eea"} Jul 01 18:37:27.053923 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.053900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fcb9x" event={"ID":"2f250d29-e5fb-4c99-8662-29dddf0e37c5","Type":"ContainerStarted","Data":"22be419f5658d02874465e7e107464f8a731cee220a94e0c244a45d8e5defca8"} Jul 01 18:37:27.057016 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.056976 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-114.ec2.internal" podStartSLOduration=22.056965213 podStartE2EDuration="22.056965213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:27.056645432 +0000 UTC m=+23.572236672" watchObservedRunningTime="2026-07-01 18:37:27.056965213 +0000 UTC m=+23.572556452" Jul 01 18:37:27.095741 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.095458 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6srdv" podStartSLOduration=3.191655813 podStartE2EDuration="23.095437298s" totalImagesPullingTime="19.903781485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:27.095100476 +0000 UTC m=+23.610691739" watchObservedRunningTime="2026-07-01 18:37:27.095437298 +0000 UTC m=+23.611028538" Jul 01 18:37:27.114067 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.114007 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b4hlx" podStartSLOduration=3.270452281 podStartE2EDuration="23.113985982s" totalImagesPullingTime="19.843533701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:27.113451809 +0000 UTC m=+23.629043049" watchObservedRunningTime="2026-07-01 18:37:27.113985982 +0000 UTC m=+23.629577222" Jul 01 18:37:27.162342 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.162289 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l8k74" podStartSLOduration=2.329384036 podStartE2EDuration="22.162272465s" totalImagesPullingTime="19.832888429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:27.132051812 +0000 UTC m=+23.647643052" watchObservedRunningTime="2026-07-01 18:37:27.162272465 +0000 UTC m=+23.677863703" Jul 01 18:37:27.179823 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.179738 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fcb9x" podStartSLOduration=7.516633431 podStartE2EDuration="23.179717385s" totalImagesPullingTime="15.663083954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:27.178967556 +0000 UTC m=+23.694558795" watchObservedRunningTime="2026-07-01 18:37:27.179717385 +0000 UTC m=+23.695308623" Jul 01 18:37:27.956074 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.956045 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:27.956222 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:27.956192 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:27.956332 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.956278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:27.956384 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:27.956351 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:27.970142 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:27.970111 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 18:37:28.059100 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.059057 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" event={"ID":"f5a15d2a32d4de4fde9197b362accbd8","Type":"ContainerStarted","Data":"413d404b03b87571ed1607c8b8b09e95f2eaf4dc82723045a86a3a598cc556d7"} Jul 01 18:37:28.060819 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.060777 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fsvrm" event={"ID":"c0aeff29-79df-4f33-88db-7a200c306c69","Type":"ContainerStarted","Data":"976b98c45bbc04c9045a1fceb9d65762a534dcbb08b434018b9faf0e92a3196d"} Jul 01 18:37:28.062803 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.062770 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" event={"ID":"3d184d95-2b81-483d-b7fd-7879bbc2de5d","Type":"ContainerStarted","Data":"8f495a33c33bf0a2d0bb990705268ee2cdade143cdd6269ae1121d05a97f20ea"} Jul 01 18:37:28.077036 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.076985 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-114.ec2.internal" podStartSLOduration=23.076964894 podStartE2EDuration="23.076964894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:28.076935332 +0000 UTC m=+24.592526575" watchObservedRunningTime="2026-07-01 18:37:28.076964894 +0000 UTC m=+24.592556134" Jul 01 18:37:28.077685 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.077640 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sh9jq" podStartSLOduration=4.269681655 podStartE2EDuration="24.077629683s" totalImagesPullingTime="19.807948028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:27.199983385 +0000 UTC m=+23.715574625" watchObservedRunningTime="2026-07-01 18:37:28.077629683 +0000 UTC m=+24.593220922" Jul 01 18:37:28.093436 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.093341 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fsvrm" podStartSLOduration=3.25790657 podStartE2EDuration="23.093320541s" totalImagesPullingTime="19.835413971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:28.092780879 +0000 UTC m=+24.608372119" watchObservedRunningTime="2026-07-01 18:37:28.093320541 +0000 UTC m=+24.608911784" Jul 01 18:37:28.949387 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.949260 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T18:37:27.970133662Z","UUID":"fae3f965-5803-4c8c-9395-2a56ccee495c","Handler":null,"Name":"","Endpoint":""} Jul 01 18:37:28.950890 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.950870 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 18:37:28.951038 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.950910 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 18:37:28.956324 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:28.956298 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:28.956454 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:28.956436 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:29.415928 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:29.415892 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:29.416550 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:29.416535 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:29.956123 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:29.956087 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:29.956408 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:29.956205 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:29.956408 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:29.956234 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:29.956408 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:29.956281 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:30.069683 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:30.069645 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" event={"ID":"3d184d95-2b81-483d-b7fd-7879bbc2de5d","Type":"ContainerStarted","Data":"7767975aec1a3d55c7d6e50448013b571a076db4f05123d7063f3176f1128fed"} Jul 01 18:37:30.072764 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:30.072741 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:37:30.073212 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:30.073182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"428f43f91a6da668d0636fa87e22382bcdec408d2d6d585131d587fd991d975f"} Jul 01 18:37:30.073304 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:30.073263 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:30.073980 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:30.073958 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l8k74" Jul 01 18:37:30.103201 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:30.103147 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c792t" podStartSLOduration=3.978253291 podStartE2EDuration="26.103131624s" totalImagesPullingTime="22.124878333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:30.102454284 +0000 UTC m=+26.618045525" watchObservedRunningTime="2026-07-01 18:37:30.103131624 +0000 UTC m=+26.618722863" Jul 01 18:37:30.956412 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:30.956377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:30.956907 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:30.956482 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:31.956747 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:31.956578 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:31.957413 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:31.956651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:31.957413 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:31.956877 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:31.957413 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:31.956940 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:32.081036 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.081010 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:37:32.081377 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.081355 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"203f9726b4c827e5bc55f7254eac71886d613e1da299a5d29bc5e18dbbb2a08f"} Jul 01 18:37:32.081804 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.081776 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:32.081933 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.081818 2562 scope.go:117] "RemoveContainer" containerID="39ebfe1191eb853d87de23010c6fdb005dca4e3d671e48a2d6b49c4b9c9fa8f2" Jul 01 18:37:32.081933 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.081863 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:32.083390 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.083364 2562 generic.go:356] "Generic (PLEG): container finished" podID="315b7908-1f18-4b3f-800d-05cf10e7da1e" containerID="d5e95aefc91e8b1a4cbf5a645300f4573795d9375c1d634f75bbcd0cb0e04ac8" exitCode=0 Jul 01 18:37:32.083487 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.083449 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerDied","Data":"d5e95aefc91e8b1a4cbf5a645300f4573795d9375c1d634f75bbcd0cb0e04ac8"} Jul 01 18:37:32.097910 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.097885 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:32.098129 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.098116 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:32.956086 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:32.956058 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:32.956274 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:32.956174 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:33.106518 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:33.106485 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:37:33.107266 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:33.107244 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" event={"ID":"264efc66-6510-4a19-94f2-908be89caa25","Type":"ContainerStarted","Data":"716e1890eee9be294f2b9101dc802efbd7a9d2957f3671110c1a3229c9909dbc"} Jul 01 18:37:33.107425 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:33.107405 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 18:37:33.138762 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:33.138664 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" podStartSLOduration=9.126433855 podStartE2EDuration="29.13864857s" totalImagesPullingTime="20.012214715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:33.137320055 +0000 UTC m=+29.652911295" watchObservedRunningTime="2026-07-01 18:37:33.13864857 +0000 UTC m=+29.654239824" Jul 01 18:37:33.957519 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:33.957255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:33.957691 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:33.957336 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:33.957691 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:33.957623 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:33.957691 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:33.957663 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:34.114613 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.114577 2562 generic.go:356] "Generic (PLEG): container finished" podID="315b7908-1f18-4b3f-800d-05cf10e7da1e" containerID="ccfcb8fccab50100ce7aa7ec0ffbab3ff30b2d58bb3a2f80bb2656c4064a19f9" exitCode=0 Jul 01 18:37:34.115085 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.114659 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerDied","Data":"ccfcb8fccab50100ce7aa7ec0ffbab3ff30b2d58bb3a2f80bb2656c4064a19f9"} Jul 01 18:37:34.115085 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.114751 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 18:37:34.146716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.146685 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j8lms"] Jul 01 18:37:34.146883 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.146794 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:34.146929 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:34.146896 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:34.150363 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.150336 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-67lhc"] Jul 01 18:37:34.150493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.150462 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:34.150590 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:34.150568 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:34.150941 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.150921 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-npbg7"] Jul 01 18:37:34.151034 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:34.151021 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:34.151136 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:34.151104 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:35.956136 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:35.956054 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:35.956568 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:35.956053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:35.956568 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:35.956167 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:35.956568 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:35.956253 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:35.956568 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:35.956053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:35.956568 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:35.956349 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:36.110170 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:36.110137 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:37:36.110339 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:36.110332 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 18:37:36.121417 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:36.121386 2562 generic.go:356] "Generic (PLEG): container finished" podID="315b7908-1f18-4b3f-800d-05cf10e7da1e" containerID="96006c0f084c872ef548d93ee4357f7707863a1114398e137ae822885aadea82" exitCode=0 Jul 01 18:37:36.121562 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:36.121426 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerDied","Data":"96006c0f084c872ef548d93ee4357f7707863a1114398e137ae822885aadea82"} Jul 01 18:37:36.125633 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:36.125514 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" podUID="264efc66-6510-4a19-94f2-908be89caa25" containerName="ovnkube-controller" probeResult="failure" output="" Jul 01 18:37:36.134124 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:36.134094 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" podUID="264efc66-6510-4a19-94f2-908be89caa25" containerName="ovnkube-controller" probeResult="failure" output="" Jul 01 18:37:37.636146 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:37.635921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:37.636567 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.636097 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:37.636567 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.636299 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs podName:d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:09.636275536 +0000 UTC m=+66.151866767 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs") pod "network-metrics-daemon-npbg7" (UID: "d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:37.737173 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:37.737088 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:37.737173 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:37.737158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:37.737403 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.737277 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:37.737403 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.737295 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:37.737403 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.737364 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret podName:a5932d17-9b81-4e7e-94ce-1db0cded279d nodeName:}" failed. No retries permitted until 2026-07-01 18:38:09.737344814 +0000 UTC m=+66.252936049 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret") pod "global-pull-secret-syncer-j8lms" (UID: "a5932d17-9b81-4e7e-94ce-1db0cded279d") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:37.737403 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.737300 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:37.737403 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.737393 2562 projected.go:196] Error preparing data for projected volume kube-api-access-2kmh8 for pod openshift-network-diagnostics/network-check-target-67lhc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:37.737658 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.737453 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8 podName:b40e8345-4927-47e8-8cd9-dba01f13a362 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:09.737435645 +0000 UTC m=+66.253026882 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kmh8" (UniqueName: "kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8") pod "network-check-target-67lhc" (UID: "b40e8345-4927-47e8-8cd9-dba01f13a362") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:37.956513 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:37.956472 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:37.956513 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:37.956505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:37.956744 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:37.956473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:37.956744 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.956608 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8lms" podUID="a5932d17-9b81-4e7e-94ce-1db0cded279d" Jul 01 18:37:37.956744 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.956664 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npbg7" podUID="d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973" Jul 01 18:37:37.956918 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:37.956770 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-67lhc" podUID="b40e8345-4927-47e8-8cd9-dba01f13a362" Jul 01 18:37:38.842559 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.842529 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-114.ec2.internal" event="NodeReady" Jul 01 18:37:38.842946 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.842663 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 18:37:38.886119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.886082 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bc86fc8cf-w7hmh"] Jul 01 18:37:38.888593 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.888567 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:38.892622 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.892597 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 18:37:38.892760 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.892657 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x2fbb\"" Jul 01 18:37:38.892820 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.892758 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 18:37:38.897035 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.897011 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 18:37:38.898425 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.898399 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 18:37:38.903351 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.903328 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bc86fc8cf-w7hmh"] Jul 01 18:37:38.923603 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.923472 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-59bd7"] Jul 01 18:37:38.925528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.925504 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wz7b8"] Jul 01 18:37:38.925857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.925818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-59bd7" Jul 01 18:37:38.927556 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.927535 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:38.928284 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.928259 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 18:37:38.928400 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.928264 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 18:37:38.928400 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.928264 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ttz69\"" Jul 01 18:37:38.928495 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.928402 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 18:37:38.929761 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.929649 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 18:37:38.929761 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.929666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 18:37:38.929955 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.929936 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7sn7m\"" Jul 01 18:37:38.938055 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.937992 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-59bd7"] Jul 01 18:37:38.938928 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:38.938907 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wz7b8"] Jul 01 18:37:39.047634 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eea556-5056-460c-809f-31ef7e9f333c-tmp-dir\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.047634 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047628 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-image-registry-private-configuration\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.047837 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-certificates\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.047837 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-bound-sa-token\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.047837 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047759 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90660e9d-a455-4cb2-8bbb-8e66428f94b9-ca-trust-extracted\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.047837 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-trusted-ca\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.048019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-installation-pull-secrets\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.048019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047934 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48k7s\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-kube-api-access-48k7s\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.048019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.047983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87032a95-9729-4094-a901-5ac3827cc3e1-cert\") pod \"ingress-canary-59bd7\" (UID: \"87032a95-9729-4094-a901-5ac3827cc3e1\") " pod="openshift-ingress-canary/ingress-canary-59bd7" Jul 01 18:37:39.048019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.048009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7qqg\" (UniqueName: \"kubernetes.io/projected/87032a95-9729-4094-a901-5ac3827cc3e1-kube-api-access-z7qqg\") pod \"ingress-canary-59bd7\" (UID: \"87032a95-9729-4094-a901-5ac3827cc3e1\") " pod="openshift-ingress-canary/ingress-canary-59bd7" Jul 01 18:37:39.048183 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.048057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgnx6\" (UniqueName: \"kubernetes.io/projected/b3eea556-5056-460c-809f-31ef7e9f333c-kube-api-access-bgnx6\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.048183 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.048104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3eea556-5056-460c-809f-31ef7e9f333c-config-volume\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.048183 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.048125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.048183 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.048162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.124059 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.124021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:39.149431 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgnx6\" (UniqueName: \"kubernetes.io/projected/b3eea556-5056-460c-809f-31ef7e9f333c-kube-api-access-bgnx6\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.149628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149442 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3eea556-5056-460c-809f-31ef7e9f333c-config-volume\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.149628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.149628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.149628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eea556-5056-460c-809f-31ef7e9f333c-tmp-dir\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.149628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-image-registry-private-configuration\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.149628 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.149589 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:37:39.149628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-certificates\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.149650 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.149674 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc86fc8cf-w7hmh: secret "image-registry-tls" not found Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.149661 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls podName:b3eea556-5056-460c-809f-31ef7e9f333c nodeName:}" failed. No retries permitted until 2026-07-01 18:37:39.649642271 +0000 UTC m=+36.165233500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls") pod "dns-default-wz7b8" (UID: "b3eea556-5056-460c-809f-31ef7e9f333c") : secret "dns-default-metrics-tls" not found Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.149748 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls podName:90660e9d-a455-4cb2-8bbb-8e66428f94b9 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:39.649730679 +0000 UTC m=+36.165321905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls") pod "image-registry-bc86fc8cf-w7hmh" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9") : secret "image-registry-tls" not found Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149801 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-bound-sa-token\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149873 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90660e9d-a455-4cb2-8bbb-8e66428f94b9-ca-trust-extracted\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-trusted-ca\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-installation-pull-secrets\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48k7s\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-kube-api-access-48k7s\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.149986 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.149975 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3eea556-5056-460c-809f-31ef7e9f333c-tmp-dir\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.150479 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.150033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87032a95-9729-4094-a901-5ac3827cc3e1-cert\") pod \"ingress-canary-59bd7\" (UID: \"87032a95-9729-4094-a901-5ac3827cc3e1\") " pod="openshift-ingress-canary/ingress-canary-59bd7" Jul 01 18:37:39.150479 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.150062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7qqg\" (UniqueName: \"kubernetes.io/projected/87032a95-9729-4094-a901-5ac3827cc3e1-kube-api-access-z7qqg\") pod \"ingress-canary-59bd7\" (UID: \"87032a95-9729-4094-a901-5ac3827cc3e1\") " pod="openshift-ingress-canary/ingress-canary-59bd7" Jul 01 18:37:39.150479 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.150234 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90660e9d-a455-4cb2-8bbb-8e66428f94b9-ca-trust-extracted\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.150479 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.150264 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3eea556-5056-460c-809f-31ef7e9f333c-config-volume\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.150479 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.150274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-certificates\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.150938 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.150916 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-trusted-ca\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.154632 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.154611 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-installation-pull-secrets\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.154732 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.154629 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-image-registry-private-configuration\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.154732 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.154654 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87032a95-9729-4094-a901-5ac3827cc3e1-cert\") pod \"ingress-canary-59bd7\" (UID: \"87032a95-9729-4094-a901-5ac3827cc3e1\") " pod="openshift-ingress-canary/ingress-canary-59bd7" Jul 01 18:37:39.162191 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.162167 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48k7s\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-kube-api-access-48k7s\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.162586 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.162562 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-bound-sa-token\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.162677 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.162577 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgnx6\" (UniqueName: \"kubernetes.io/projected/b3eea556-5056-460c-809f-31ef7e9f333c-kube-api-access-bgnx6\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.162737 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.162695 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7qqg\" (UniqueName: \"kubernetes.io/projected/87032a95-9729-4094-a901-5ac3827cc3e1-kube-api-access-z7qqg\") pod \"ingress-canary-59bd7\" (UID: \"87032a95-9729-4094-a901-5ac3827cc3e1\") " pod="openshift-ingress-canary/ingress-canary-59bd7" Jul 01 18:37:39.226180 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.226135 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:39.239446 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.239414 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-59bd7" Jul 01 18:37:39.654959 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.654915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:39.655135 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.655027 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:39.655135 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.655103 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:37:39.655135 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.655123 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc86fc8cf-w7hmh: secret "image-registry-tls" not found Jul 01 18:37:39.655302 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.655137 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:37:39.655302 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.655188 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls podName:b3eea556-5056-460c-809f-31ef7e9f333c nodeName:}" failed. No retries permitted until 2026-07-01 18:37:40.655175504 +0000 UTC m=+37.170766721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls") pod "dns-default-wz7b8" (UID: "b3eea556-5056-460c-809f-31ef7e9f333c") : secret "dns-default-metrics-tls" not found Jul 01 18:37:39.655302 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:39.655201 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls podName:90660e9d-a455-4cb2-8bbb-8e66428f94b9 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:40.655195146 +0000 UTC m=+37.170786363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls") pod "image-registry-bc86fc8cf-w7hmh" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9") : secret "image-registry-tls" not found Jul 01 18:37:39.956392 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.956363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:37:39.957122 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.956398 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:37:39.957122 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.956366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:37:39.959598 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.959575 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 18:37:39.959746 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.959597 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 18:37:39.959746 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.959667 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 18:37:39.960586 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.960569 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lwv7s\"" Jul 01 18:37:39.960681 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.960570 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2vz9\"" Jul 01 18:37:39.960681 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:39.960575 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 18:37:40.095231 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:40.095044 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-59bd7"] Jul 01 18:37:40.099602 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:37:40.099580 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87032a95_9729_4094_a901_5ac3827cc3e1.slice/crio-2c41898ec67cbf228f04fe21f713eeac72b2c3d58faff6ecc92535357f2a3515 WatchSource:0}: Error finding container 2c41898ec67cbf228f04fe21f713eeac72b2c3d58faff6ecc92535357f2a3515: Status 404 returned error can't find the container with id 2c41898ec67cbf228f04fe21f713eeac72b2c3d58faff6ecc92535357f2a3515 Jul 01 18:37:40.134584 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:40.134545 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerStarted","Data":"e26e6994e58495fe6224f61765d02330392fe48d83e7212120121a2077cb6750"} Jul 01 18:37:40.135783 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:40.135755 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-59bd7" event={"ID":"87032a95-9729-4094-a901-5ac3827cc3e1","Type":"ContainerStarted","Data":"2c41898ec67cbf228f04fe21f713eeac72b2c3d58faff6ecc92535357f2a3515"} Jul 01 18:37:40.662965 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:40.662922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:40.663160 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:40.662976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:40.663160 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:40.663088 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:37:40.663160 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:40.663096 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:37:40.663345 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:40.663103 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc86fc8cf-w7hmh: secret "image-registry-tls" not found Jul 01 18:37:40.663345 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:40.663182 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls podName:b3eea556-5056-460c-809f-31ef7e9f333c nodeName:}" failed. No retries permitted until 2026-07-01 18:37:42.663157588 +0000 UTC m=+39.178748839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls") pod "dns-default-wz7b8" (UID: "b3eea556-5056-460c-809f-31ef7e9f333c") : secret "dns-default-metrics-tls" not found Jul 01 18:37:40.663345 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:40.663227 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls podName:90660e9d-a455-4cb2-8bbb-8e66428f94b9 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:42.663211378 +0000 UTC m=+39.178802615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls") pod "image-registry-bc86fc8cf-w7hmh" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9") : secret "image-registry-tls" not found Jul 01 18:37:41.140975 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:41.140937 2562 generic.go:356] "Generic (PLEG): container finished" podID="315b7908-1f18-4b3f-800d-05cf10e7da1e" containerID="e26e6994e58495fe6224f61765d02330392fe48d83e7212120121a2077cb6750" exitCode=0 Jul 01 18:37:41.141565 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:41.140995 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerDied","Data":"e26e6994e58495fe6224f61765d02330392fe48d83e7212120121a2077cb6750"} Jul 01 18:37:42.145769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:42.145736 2562 generic.go:356] "Generic (PLEG): container finished" podID="315b7908-1f18-4b3f-800d-05cf10e7da1e" containerID="66cbf43971119462aa7fcba090658fe608f533d6dc7b9eca51b4ad8b833dad69" exitCode=0 Jul 01 18:37:42.146152 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:42.145792 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerDied","Data":"66cbf43971119462aa7fcba090658fe608f533d6dc7b9eca51b4ad8b833dad69"} Jul 01 18:37:42.678254 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:42.678216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:42.678449 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:42.678380 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:37:42.678449 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:42.678398 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc86fc8cf-w7hmh: secret "image-registry-tls" not found Jul 01 18:37:42.678449 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:42.678402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:42.678449 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:42.678450 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls podName:90660e9d-a455-4cb2-8bbb-8e66428f94b9 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:46.678433939 +0000 UTC m=+43.194025158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls") pod "image-registry-bc86fc8cf-w7hmh" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9") : secret "image-registry-tls" not found Jul 01 18:37:42.678657 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:42.678497 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:37:42.678657 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:42.678534 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls podName:b3eea556-5056-460c-809f-31ef7e9f333c nodeName:}" failed. No retries permitted until 2026-07-01 18:37:46.678523751 +0000 UTC m=+43.194114968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls") pod "dns-default-wz7b8" (UID: "b3eea556-5056-460c-809f-31ef7e9f333c") : secret "dns-default-metrics-tls" not found Jul 01 18:37:43.151701 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:43.151650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9995" event={"ID":"315b7908-1f18-4b3f-800d-05cf10e7da1e","Type":"ContainerStarted","Data":"074f09259ef9e4e3eab7ceb93507fe925996de59e4d45746aa38baefdfcd5fde"} Jul 01 18:37:43.153051 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:43.153028 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-59bd7" event={"ID":"87032a95-9729-4094-a901-5ac3827cc3e1","Type":"ContainerStarted","Data":"b9627b592a021f51d78bfc34ad6b8300718a70b8650ff58d8c9f5f120fd1563d"} Jul 01 18:37:43.179707 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:43.179644 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t9995" podStartSLOduration=8.375677477 podStartE2EDuration="39.179628962s" totalImagesPullingTime="30.803951485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:43.178445329 +0000 UTC m=+39.694036567" watchObservedRunningTime="2026-07-01 18:37:43.179628962 +0000 UTC m=+39.695220200" Jul 01 18:37:43.195955 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:43.195899 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-59bd7" podStartSLOduration=2.8229123659999997 podStartE2EDuration="5.195881155s" totalImagesPullingTime="2.372968789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:43.195533793 +0000 UTC m=+39.711125031" watchObservedRunningTime="2026-07-01 18:37:43.195881155 +0000 UTC m=+39.711472397" Jul 01 18:37:43.328856 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:43.328803 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:43.421762 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:43.421668 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:44.593387 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:44.593208 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:44.681062 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:44.681031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:45.163237 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:45.163200 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:45.256505 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:45.256461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:46.709708 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:46.709669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:46.709708 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:46.709713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:46.710152 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:46.709831 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:37:46.710152 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:46.709860 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:37:46.710152 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:46.709873 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc86fc8cf-w7hmh: secret "image-registry-tls" not found Jul 01 18:37:46.710152 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:46.709917 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls podName:b3eea556-5056-460c-809f-31ef7e9f333c nodeName:}" failed. No retries permitted until 2026-07-01 18:37:54.709900045 +0000 UTC m=+51.225491266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls") pod "dns-default-wz7b8" (UID: "b3eea556-5056-460c-809f-31ef7e9f333c") : secret "dns-default-metrics-tls" not found Jul 01 18:37:46.710152 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:46.709931 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls podName:90660e9d-a455-4cb2-8bbb-8e66428f94b9 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:54.7099253 +0000 UTC m=+51.225516517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls") pod "image-registry-bc86fc8cf-w7hmh" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9") : secret "image-registry-tls" not found Jul 01 18:37:54.769964 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:54.769920 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:37:54.770447 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:37:54.770013 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:37:54.770447 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:54.770049 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:37:54.770447 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:54.770075 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc86fc8cf-w7hmh: secret "image-registry-tls" not found Jul 01 18:37:54.770447 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:54.770094 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:37:54.770447 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:54.770155 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls podName:90660e9d-a455-4cb2-8bbb-8e66428f94b9 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:10.770132223 +0000 UTC m=+67.285723457 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls") pod "image-registry-bc86fc8cf-w7hmh" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9") : secret "image-registry-tls" not found Jul 01 18:37:54.770447 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:37:54.770175 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls podName:b3eea556-5056-460c-809f-31ef7e9f333c nodeName:}" failed. No retries permitted until 2026-07-01 18:38:10.770166638 +0000 UTC m=+67.285757855 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls") pod "dns-default-wz7b8" (UID: "b3eea556-5056-460c-809f-31ef7e9f333c") : secret "dns-default-metrics-tls" not found Jul 01 18:38:06.135445 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:06.135411 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d9nnh" Jul 01 18:38:09.679256 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.679215 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:38:09.681764 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.681745 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 18:38:09.689556 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:09.689539 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 18:38:09.689625 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:09.689606 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs podName:d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973 nodeName:}" failed. No retries permitted until 2026-07-01 18:39:13.689591311 +0000 UTC m=+130.205182528 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs") pod "network-metrics-daemon-npbg7" (UID: "d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973") : secret "metrics-daemon-secret" not found Jul 01 18:38:09.779737 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.779700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:38:09.779811 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.779757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:38:09.783348 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.783325 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 18:38:09.784258 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.784239 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 18:38:09.792766 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.792742 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 18:38:09.793769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.793748 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5932d17-9b81-4e7e-94ce-1db0cded279d-original-pull-secret\") pod \"global-pull-secret-syncer-j8lms\" (UID: \"a5932d17-9b81-4e7e-94ce-1db0cded279d\") " pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:38:09.802924 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.802904 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kmh8\" (UniqueName: \"kubernetes.io/projected/b40e8345-4927-47e8-8cd9-dba01f13a362-kube-api-access-2kmh8\") pod \"network-check-target-67lhc\" (UID: \"b40e8345-4927-47e8-8cd9-dba01f13a362\") " pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:38:09.971364 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.971292 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lwv7s\"" Jul 01 18:38:09.978698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.978681 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:38:09.980877 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:09.980860 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8lms" Jul 01 18:38:10.113367 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:10.113337 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j8lms"] Jul 01 18:38:10.116991 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:10.116961 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5932d17_9b81_4e7e_94ce_1db0cded279d.slice/crio-676bfbd2c97746d91f603daf56d7d967e21cec8aa25999202de1f6681ca02186 WatchSource:0}: Error finding container 676bfbd2c97746d91f603daf56d7d967e21cec8aa25999202de1f6681ca02186: Status 404 returned error can't find the container with id 676bfbd2c97746d91f603daf56d7d967e21cec8aa25999202de1f6681ca02186 Jul 01 18:38:10.127076 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:10.127052 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-67lhc"] Jul 01 18:38:10.129849 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:10.129817 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40e8345_4927_47e8_8cd9_dba01f13a362.slice/crio-7620d98b6fd0d4e3df83609e034daf063e19c9ff7bab9fb802812a825717c4bd WatchSource:0}: Error finding container 7620d98b6fd0d4e3df83609e034daf063e19c9ff7bab9fb802812a825717c4bd: Status 404 returned error can't find the container with id 7620d98b6fd0d4e3df83609e034daf063e19c9ff7bab9fb802812a825717c4bd Jul 01 18:38:10.212944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:10.212908 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j8lms" event={"ID":"a5932d17-9b81-4e7e-94ce-1db0cded279d","Type":"ContainerStarted","Data":"676bfbd2c97746d91f603daf56d7d967e21cec8aa25999202de1f6681ca02186"} Jul 01 18:38:10.213964 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:10.213941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-67lhc" event={"ID":"b40e8345-4927-47e8-8cd9-dba01f13a362","Type":"ContainerStarted","Data":"7620d98b6fd0d4e3df83609e034daf063e19c9ff7bab9fb802812a825717c4bd"} Jul 01 18:38:10.787740 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:10.787677 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:38:10.788232 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:10.787779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:38:10.788232 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:10.787867 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:38:10.788232 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:10.787891 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bc86fc8cf-w7hmh: secret "image-registry-tls" not found Jul 01 18:38:10.788232 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:10.787930 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:38:10.788232 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:10.787962 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls podName:90660e9d-a455-4cb2-8bbb-8e66428f94b9 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:42.787939806 +0000 UTC m=+99.303531052 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls") pod "image-registry-bc86fc8cf-w7hmh" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9") : secret "image-registry-tls" not found Jul 01 18:38:10.788232 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:10.787985 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls podName:b3eea556-5056-460c-809f-31ef7e9f333c nodeName:}" failed. No retries permitted until 2026-07-01 18:38:42.787973631 +0000 UTC m=+99.303564852 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls") pod "dns-default-wz7b8" (UID: "b3eea556-5056-460c-809f-31ef7e9f333c") : secret "dns-default-metrics-tls" not found Jul 01 18:38:15.228809 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:15.228771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-67lhc" event={"ID":"b40e8345-4927-47e8-8cd9-dba01f13a362","Type":"ContainerStarted","Data":"39b4b2c12615af5afff32eb75a0510b117655a423efb16a75e98075379f4d56c"} Jul 01 18:38:15.229292 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:15.228891 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:38:15.230166 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:15.230145 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j8lms" event={"ID":"a5932d17-9b81-4e7e-94ce-1db0cded279d","Type":"ContainerStarted","Data":"fc08c9b068fa8bf261186741e0ceb3bb0abd972b320229c66ae87661343b6d21"} Jul 01 18:38:15.246828 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:15.246786 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-67lhc" podStartSLOduration=66.805995918 podStartE2EDuration="1m11.246773526s" totalImagesPullingTime="4.440777608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:15.245952597 +0000 UTC m=+71.761543837" watchObservedRunningTime="2026-07-01 18:38:15.246773526 +0000 UTC m=+71.762364764" Jul 01 18:38:24.392411 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.392356 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j8lms" podStartSLOduration=74.935050489 podStartE2EDuration="1m19.392334579s" totalImagesPullingTime="4.45728409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:15.266258339 +0000 UTC m=+71.781849578" watchObservedRunningTime="2026-07-01 18:38:24.392334579 +0000 UTC m=+80.907925819" Jul 01 18:38:24.392981 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.392948 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-lfn8v"] Jul 01 18:38:24.394738 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.394723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.397095 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.397071 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7f7dl\"" Jul 01 18:38:24.397212 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.397079 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:38:24.397698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.397684 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 01 18:38:24.397853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.397822 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 01 18:38:24.398062 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.397895 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 01 18:38:24.401493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.401473 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 01 18:38:24.408282 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.408263 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8"] Jul 01 18:38:24.410080 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.410066 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8" Jul 01 18:38:24.412650 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.412631 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:38:24.412753 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.412687 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-726d5\"" Jul 01 18:38:24.413169 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.413144 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 01 18:38:24.417377 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.417354 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-lfn8v"] Jul 01 18:38:24.425338 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.425315 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8"] Jul 01 18:38:24.484498 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.484460 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f8b9fef-153e-4b4c-b13d-43ac17462991-trusted-ca\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.484690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.484548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f8b9fef-153e-4b4c-b13d-43ac17462991-config\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.484690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.484590 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrdhr\" (UniqueName: \"kubernetes.io/projected/7f8b9fef-153e-4b4c-b13d-43ac17462991-kube-api-access-hrdhr\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.484690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.484614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f8b9fef-153e-4b4c-b13d-43ac17462991-serving-cert\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.484690 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.484666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf2vt\" (UniqueName: \"kubernetes.io/projected/50112456-daf6-4609-99d3-87004be60195-kube-api-access-rf2vt\") pod \"volume-data-source-validator-69d9f679db-g4gg8\" (UID: \"50112456-daf6-4609-99d3-87004be60195\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8" Jul 01 18:38:24.585015 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.584965 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f8b9fef-153e-4b4c-b13d-43ac17462991-config\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.585206 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.585034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrdhr\" (UniqueName: \"kubernetes.io/projected/7f8b9fef-153e-4b4c-b13d-43ac17462991-kube-api-access-hrdhr\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.585206 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.585061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f8b9fef-153e-4b4c-b13d-43ac17462991-serving-cert\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.585206 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.585106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf2vt\" (UniqueName: \"kubernetes.io/projected/50112456-daf6-4609-99d3-87004be60195-kube-api-access-rf2vt\") pod \"volume-data-source-validator-69d9f679db-g4gg8\" (UID: \"50112456-daf6-4609-99d3-87004be60195\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8" Jul 01 18:38:24.585206 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.585136 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f8b9fef-153e-4b4c-b13d-43ac17462991-trusted-ca\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.585742 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.585717 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f8b9fef-153e-4b4c-b13d-43ac17462991-config\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.586075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.586052 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f8b9fef-153e-4b4c-b13d-43ac17462991-trusted-ca\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.587392 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.587370 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f8b9fef-153e-4b4c-b13d-43ac17462991-serving-cert\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.606401 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.606370 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrdhr\" (UniqueName: \"kubernetes.io/projected/7f8b9fef-153e-4b4c-b13d-43ac17462991-kube-api-access-hrdhr\") pod \"console-operator-7fdbdf9c99-lfn8v\" (UID: \"7f8b9fef-153e-4b4c-b13d-43ac17462991\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.608631 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.608606 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf2vt\" (UniqueName: \"kubernetes.io/projected/50112456-daf6-4609-99d3-87004be60195-kube-api-access-rf2vt\") pod \"volume-data-source-validator-69d9f679db-g4gg8\" (UID: \"50112456-daf6-4609-99d3-87004be60195\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8" Jul 01 18:38:24.704013 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.703922 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:24.721941 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.721913 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8" Jul 01 18:38:24.837931 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.837899 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-lfn8v"] Jul 01 18:38:24.840998 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:24.840972 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8b9fef_153e_4b4c_b13d_43ac17462991.slice/crio-2600a7d670351c33740981e08651f5ee3b36f2153d815b0b9eb3f798cd487830 WatchSource:0}: Error finding container 2600a7d670351c33740981e08651f5ee3b36f2153d815b0b9eb3f798cd487830: Status 404 returned error can't find the container with id 2600a7d670351c33740981e08651f5ee3b36f2153d815b0b9eb3f798cd487830 Jul 01 18:38:24.853116 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:24.853095 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8"] Jul 01 18:38:24.856091 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:24.856069 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50112456_daf6_4609_99d3_87004be60195.slice/crio-33b1a87a9bdd917091e5a2bfaef35af3a72395224d52d87683b80ed46951c865 WatchSource:0}: Error finding container 33b1a87a9bdd917091e5a2bfaef35af3a72395224d52d87683b80ed46951c865: Status 404 returned error can't find the container with id 33b1a87a9bdd917091e5a2bfaef35af3a72395224d52d87683b80ed46951c865 Jul 01 18:38:25.254781 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:25.254652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" event={"ID":"7f8b9fef-153e-4b4c-b13d-43ac17462991","Type":"ContainerStarted","Data":"2600a7d670351c33740981e08651f5ee3b36f2153d815b0b9eb3f798cd487830"} Jul 01 18:38:25.255699 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:25.255675 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8" event={"ID":"50112456-daf6-4609-99d3-87004be60195","Type":"ContainerStarted","Data":"33b1a87a9bdd917091e5a2bfaef35af3a72395224d52d87683b80ed46951c865"} Jul 01 18:38:26.259295 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:26.259204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8" event={"ID":"50112456-daf6-4609-99d3-87004be60195","Type":"ContainerStarted","Data":"6e92c368accc30570096c568157331fde27ff3b5634db73b666a095fe8ffafef"} Jul 01 18:38:26.282594 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:26.282444 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g4gg8" podStartSLOduration=1.286860603 podStartE2EDuration="2.282391588s" totalImagesPullingTime="995.530985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:26.282101508 +0000 UTC m=+82.797692771" watchObservedRunningTime="2026-07-01 18:38:26.282391588 +0000 UTC m=+82.797982827" Jul 01 18:38:27.219215 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.219141 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rzrfw"] Jul 01 18:38:27.221089 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.221074 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.223397 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.223376 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 18:38:27.223495 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.223386 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 18:38:27.223959 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.223944 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 18:38:27.224019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.223976 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q4zk9\"" Jul 01 18:38:27.224063 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.223975 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 18:38:27.247275 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.247248 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rzrfw"] Jul 01 18:38:27.264115 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.264093 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/0.log" Jul 01 18:38:27.264459 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.264129 2562 generic.go:356] "Generic (PLEG): container finished" podID="7f8b9fef-153e-4b4c-b13d-43ac17462991" containerID="bb621dc8a408cba47ad993c512c92103ae8669cab2a60043cd6bd2b3d120b892" exitCode=255 Jul 01 18:38:27.264459 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.264209 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" event={"ID":"7f8b9fef-153e-4b4c-b13d-43ac17462991","Type":"ContainerDied","Data":"bb621dc8a408cba47ad993c512c92103ae8669cab2a60043cd6bd2b3d120b892"} Jul 01 18:38:27.264459 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.264419 2562 scope.go:117] "RemoveContainer" containerID="bb621dc8a408cba47ad993c512c92103ae8669cab2a60043cd6bd2b3d120b892" Jul 01 18:38:27.305830 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.305805 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.305975 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.305859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwk64\" (UniqueName: \"kubernetes.io/projected/99cabad8-b09b-48f1-a331-7325be9882c0-kube-api-access-pwk64\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.306060 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.306025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.306692 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.306662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99cabad8-b09b-48f1-a331-7325be9882c0-crio-socket\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.306986 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.306711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99cabad8-b09b-48f1-a331-7325be9882c0-data-volume\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.407504 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.407461 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwk64\" (UniqueName: \"kubernetes.io/projected/99cabad8-b09b-48f1-a331-7325be9882c0-kube-api-access-pwk64\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.407647 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.407517 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.407647 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:27.407582 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:27.407753 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:27.407648 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:27.907633992 +0000 UTC m=+84.423225210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : configmap "kube-rbac-proxy" not found Jul 01 18:38:27.407753 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.407739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99cabad8-b09b-48f1-a331-7325be9882c0-crio-socket\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.407837 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.407770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99cabad8-b09b-48f1-a331-7325be9882c0-data-volume\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.407901 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.407832 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.407901 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.407868 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/99cabad8-b09b-48f1-a331-7325be9882c0-crio-socket\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.407978 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:27.407949 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:27.408020 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:27.407990 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:27.907979948 +0000 UTC m=+84.423571165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:27.408131 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.408115 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/99cabad8-b09b-48f1-a331-7325be9882c0-data-volume\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.418066 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.418040 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwk64\" (UniqueName: \"kubernetes.io/projected/99cabad8-b09b-48f1-a331-7325be9882c0-kube-api-access-pwk64\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.911469 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.911414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.911656 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:27.911562 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:27.911656 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:27.911582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:27.911656 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:27.911623 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:28.911609643 +0000 UTC m=+85.427200859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:27.911656 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:27.911650 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:27.911796 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:27.911691 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:28.911679908 +0000 UTC m=+85.427271124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : configmap "kube-rbac-proxy" not found Jul 01 18:38:28.268514 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:28.268446 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:38:28.268926 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:28.268855 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/0.log" Jul 01 18:38:28.268926 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:28.268888 2562 generic.go:356] "Generic (PLEG): container finished" podID="7f8b9fef-153e-4b4c-b13d-43ac17462991" containerID="baf59fd90a496eea40a6cafa527307c98233968a04ece4366aa1ba9c148da0fe" exitCode=255 Jul 01 18:38:28.269010 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:28.268954 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" event={"ID":"7f8b9fef-153e-4b4c-b13d-43ac17462991","Type":"ContainerDied","Data":"baf59fd90a496eea40a6cafa527307c98233968a04ece4366aa1ba9c148da0fe"} Jul 01 18:38:28.269060 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:28.269021 2562 scope.go:117] "RemoveContainer" containerID="bb621dc8a408cba47ad993c512c92103ae8669cab2a60043cd6bd2b3d120b892" Jul 01 18:38:28.269239 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:28.269219 2562 scope.go:117] "RemoveContainer" containerID="baf59fd90a496eea40a6cafa527307c98233968a04ece4366aa1ba9c148da0fe" Jul 01 18:38:28.269414 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:28.269397 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-lfn8v_openshift-console-operator(7f8b9fef-153e-4b4c-b13d-43ac17462991)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" podUID="7f8b9fef-153e-4b4c-b13d-43ac17462991" Jul 01 18:38:28.917969 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:28.917930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:28.918148 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:28.917982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:28.918148 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:28.918050 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:28.918148 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:28.918085 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:28.918148 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:28.918104 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:30.918085492 +0000 UTC m=+87.433676709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : configmap "kube-rbac-proxy" not found Jul 01 18:38:28.918148 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:28.918140 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:30.918125937 +0000 UTC m=+87.433717154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:29.008386 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.008359 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fcb9x_2f250d29-e5fb-4c99-8662-29dddf0e37c5/dns-node-resolver/0.log" Jul 01 18:38:29.273742 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.273660 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:38:29.274164 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.274000 2562 scope.go:117] "RemoveContainer" containerID="baf59fd90a496eea40a6cafa527307c98233968a04ece4366aa1ba9c148da0fe" Jul 01 18:38:29.274164 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:29.274154 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-lfn8v_openshift-console-operator(7f8b9fef-153e-4b4c-b13d-43ac17462991)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" podUID="7f8b9fef-153e-4b4c-b13d-43ac17462991" Jul 01 18:38:29.955195 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.955158 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6"] Jul 01 18:38:29.957935 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.957910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" Jul 01 18:38:29.960262 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.960237 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 01 18:38:29.960971 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.960938 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-sblpw\"" Jul 01 18:38:29.961095 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.961008 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 01 18:38:29.978284 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:29.978260 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6"] Jul 01 18:38:30.026103 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.026061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-859bv\" (UniqueName: \"kubernetes.io/projected/9e391929-8cd1-4e44-a07f-724feb9977b5-kube-api-access-859bv\") pod \"migrator-786f9fbc5-kvzr6\" (UID: \"9e391929-8cd1-4e44-a07f-724feb9977b5\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" Jul 01 18:38:30.127236 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.127195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-859bv\" (UniqueName: \"kubernetes.io/projected/9e391929-8cd1-4e44-a07f-724feb9977b5-kube-api-access-859bv\") pod \"migrator-786f9fbc5-kvzr6\" (UID: \"9e391929-8cd1-4e44-a07f-724feb9977b5\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" Jul 01 18:38:30.140570 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.140540 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-859bv\" (UniqueName: \"kubernetes.io/projected/9e391929-8cd1-4e44-a07f-724feb9977b5-kube-api-access-859bv\") pod \"migrator-786f9fbc5-kvzr6\" (UID: \"9e391929-8cd1-4e44-a07f-724feb9977b5\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" Jul 01 18:38:30.208560 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.208488 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sh9jq_e4c2815f-c0b3-4524-b6ce-96d3c5aba855/node-ca/0.log" Jul 01 18:38:30.266740 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.266706 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" Jul 01 18:38:30.401989 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.401957 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6"] Jul 01 18:38:30.405097 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:30.405073 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e391929_8cd1_4e44_a07f_724feb9977b5.slice/crio-be554cee745fb3f12ac92deaa02045e45060d95a0553e960fd8bc68d9894b089 WatchSource:0}: Error finding container be554cee745fb3f12ac92deaa02045e45060d95a0553e960fd8bc68d9894b089: Status 404 returned error can't find the container with id be554cee745fb3f12ac92deaa02045e45060d95a0553e960fd8bc68d9894b089 Jul 01 18:38:30.606985 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.606946 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-59bd7_87032a95-9729-4094-a901-5ac3827cc3e1/serve-healthcheck-canary/0.log" Jul 01 18:38:30.934196 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.934109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:30.934366 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:30.934224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:30.934366 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:30.934288 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:30.934366 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:30.934347 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:30.934533 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:30.934386 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:34.934363697 +0000 UTC m=+91.449954922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : configmap "kube-rbac-proxy" not found Jul 01 18:38:30.934533 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:30.934410 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:34.934393305 +0000 UTC m=+91.449984538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:31.282866 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:31.282752 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" event={"ID":"9e391929-8cd1-4e44-a07f-724feb9977b5","Type":"ContainerStarted","Data":"be554cee745fb3f12ac92deaa02045e45060d95a0553e960fd8bc68d9894b089"} Jul 01 18:38:32.287450 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:32.287405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" event={"ID":"9e391929-8cd1-4e44-a07f-724feb9977b5","Type":"ContainerStarted","Data":"06366da0d74e805652623f4955475c5fd0431543530efab631791b985152d006"} Jul 01 18:38:32.287868 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:32.287473 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" event={"ID":"9e391929-8cd1-4e44-a07f-724feb9977b5","Type":"ContainerStarted","Data":"089184b7fbd21dfa60fd007bdade2042f881136e2f33b28248a3eca670cb6f2e"} Jul 01 18:38:32.307931 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:32.307882 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-kvzr6" podStartSLOduration=2.247628503 podStartE2EDuration="3.307866432s" totalImagesPullingTime="1.060237929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:32.307632275 +0000 UTC m=+88.823223515" watchObservedRunningTime="2026-07-01 18:38:32.307866432 +0000 UTC m=+88.823457649" Jul 01 18:38:34.704976 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:34.704928 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:34.705339 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:34.705006 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:34.705339 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:34.705311 2562 scope.go:117] "RemoveContainer" containerID="baf59fd90a496eea40a6cafa527307c98233968a04ece4366aa1ba9c148da0fe" Jul 01 18:38:34.705474 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:34.705454 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-lfn8v_openshift-console-operator(7f8b9fef-153e-4b4c-b13d-43ac17462991)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" podUID="7f8b9fef-153e-4b4c-b13d-43ac17462991" Jul 01 18:38:34.964101 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:34.963998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:34.964101 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:34.964106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:34.964308 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:34.964126 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:34.964308 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:34.964185 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:42.96416994 +0000 UTC m=+99.479761161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : configmap "kube-rbac-proxy" not found Jul 01 18:38:34.964308 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:34.964193 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:34.964308 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:34.964231 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:42.964220439 +0000 UTC m=+99.479811655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:42.821725 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:42.821688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:38:42.822129 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:42.821739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:38:42.823972 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:42.823949 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3eea556-5056-460c-809f-31ef7e9f333c-metrics-tls\") pod \"dns-default-wz7b8\" (UID: \"b3eea556-5056-460c-809f-31ef7e9f333c\") " pod="openshift-dns/dns-default-wz7b8" Jul 01 18:38:42.824196 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:42.824175 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"image-registry-bc86fc8cf-w7hmh\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:38:42.849866 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:42.849827 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7sn7m\"" Jul 01 18:38:42.855497 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:42.855483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wz7b8" Jul 01 18:38:42.992130 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:42.992098 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wz7b8"] Jul 01 18:38:42.995515 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:42.995490 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3eea556_5056_460c_809f_31ef7e9f333c.slice/crio-634d41d7cb447e1e708ba37cd6d9468acedea26bc2c1ea206c345c8a1e7133df WatchSource:0}: Error finding container 634d41d7cb447e1e708ba37cd6d9468acedea26bc2c1ea206c345c8a1e7133df: Status 404 returned error can't find the container with id 634d41d7cb447e1e708ba37cd6d9468acedea26bc2c1ea206c345c8a1e7133df Jul 01 18:38:43.023382 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:43.023355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:43.023501 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:43.023420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:43.023569 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:43.023522 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:43.023618 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:43.023610 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls podName:99cabad8-b09b-48f1-a331-7325be9882c0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:59.023587479 +0000 UTC m=+115.539178714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzrfw" (UID: "99cabad8-b09b-48f1-a331-7325be9882c0") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:43.023889 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:43.023871 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/99cabad8-b09b-48f1-a331-7325be9882c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:43.104509 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:43.104482 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x2fbb\"" Jul 01 18:38:43.111104 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:43.111082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:38:43.287586 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:43.287556 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bc86fc8cf-w7hmh"] Jul 01 18:38:43.290607 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:43.290581 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90660e9d_a455_4cb2_8bbb_8e66428f94b9.slice/crio-d353398c84b7053d93d2dec83f57f1f8e579739db98c776e11c87ea2c924702a WatchSource:0}: Error finding container d353398c84b7053d93d2dec83f57f1f8e579739db98c776e11c87ea2c924702a: Status 404 returned error can't find the container with id d353398c84b7053d93d2dec83f57f1f8e579739db98c776e11c87ea2c924702a Jul 01 18:38:43.324154 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:43.324125 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wz7b8" event={"ID":"b3eea556-5056-460c-809f-31ef7e9f333c","Type":"ContainerStarted","Data":"634d41d7cb447e1e708ba37cd6d9468acedea26bc2c1ea206c345c8a1e7133df"} Jul 01 18:38:43.325545 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:43.325518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" event={"ID":"90660e9d-a455-4cb2-8bbb-8e66428f94b9","Type":"ContainerStarted","Data":"d353398c84b7053d93d2dec83f57f1f8e579739db98c776e11c87ea2c924702a"} Jul 01 18:38:44.330813 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:44.330763 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" event={"ID":"90660e9d-a455-4cb2-8bbb-8e66428f94b9","Type":"ContainerStarted","Data":"961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd"} Jul 01 18:38:44.331272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:44.330895 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:38:45.335869 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:45.335820 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wz7b8" event={"ID":"b3eea556-5056-460c-809f-31ef7e9f333c","Type":"ContainerStarted","Data":"bd0df9aaa8c4803ee795bccdbe024c85bd916695895c2f48a9bebb566e0992f1"} Jul 01 18:38:45.336330 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:45.335892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wz7b8" event={"ID":"b3eea556-5056-460c-809f-31ef7e9f333c","Type":"ContainerStarted","Data":"391de09e1de523f675804ad5db7ac778957f86eec99d21529a64c46ce7cdd88b"} Jul 01 18:38:45.369517 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:45.369465 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" podStartSLOduration=100.369446627 podStartE2EDuration="1m40.369446627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:44.384809162 +0000 UTC m=+100.900400403" watchObservedRunningTime="2026-07-01 18:38:45.369446627 +0000 UTC m=+101.885037868" Jul 01 18:38:45.370037 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:45.370007 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wz7b8" podStartSLOduration=65.816428297 podStartE2EDuration="1m7.369998408s" totalImagesPullingTime="1.553570111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:45.369391703 +0000 UTC m=+101.884982941" watchObservedRunningTime="2026-07-01 18:38:45.369998408 +0000 UTC m=+101.885589646" Jul 01 18:38:46.235282 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:46.235251 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-67lhc" Jul 01 18:38:46.339165 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:46.339137 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wz7b8" Jul 01 18:38:47.956944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:47.956912 2562 scope.go:117] "RemoveContainer" containerID="baf59fd90a496eea40a6cafa527307c98233968a04ece4366aa1ba9c148da0fe" Jul 01 18:38:48.348198 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:48.348120 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:38:48.348198 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:48.348183 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" event={"ID":"7f8b9fef-153e-4b4c-b13d-43ac17462991","Type":"ContainerStarted","Data":"fa92536512ad6e06fdea374a0808ea4fca43051cbc135a712411409d83eab151"} Jul 01 18:38:48.348497 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:48.348463 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:48.812362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:48.812329 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" Jul 01 18:38:48.844197 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:48.844145 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-lfn8v" podStartSLOduration=22.756392099 podStartE2EDuration="24.844127836s" totalImagesPullingTime="2.087735737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:48.372976814 +0000 UTC m=+104.888568063" watchObservedRunningTime="2026-07-01 18:38:48.844127836 +0000 UTC m=+105.359719109" Jul 01 18:38:55.260234 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.260196 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd"] Jul 01 18:38:55.262513 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.262482 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" Jul 01 18:38:55.272119 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.272097 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 01 18:38:55.273196 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.273167 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 01 18:38:55.273319 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.273276 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 01 18:38:55.273497 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.273480 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 01 18:38:55.277836 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:38:55.277808 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"managed-serviceaccount-dockercfg-vkx98\" is forbidden: User \"system:node:ip-10-0-134-114.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-134-114.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vkx98\"" type="*v1.Secret" Jul 01 18:38:55.302612 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.302589 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd"] Jul 01 18:38:55.357304 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.357272 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx"] Jul 01 18:38:55.359095 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.359080 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" Jul 01 18:38:55.361682 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.361663 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 01 18:38:55.361915 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.361902 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 01 18:38:55.361982 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.361937 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8m5h7\"" Jul 01 18:38:55.376128 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.376102 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx"] Jul 01 18:38:55.422000 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.421969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d73f07cc-dce9-4668-968f-14d00c2e1360-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8579cbc946-5d5qd\" (UID: \"d73f07cc-dce9-4668-968f-14d00c2e1360\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" Jul 01 18:38:55.422160 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.422025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6wkd\" (UniqueName: \"kubernetes.io/projected/d73f07cc-dce9-4668-968f-14d00c2e1360-kube-api-access-h6wkd\") pod \"managed-serviceaccount-addon-agent-8579cbc946-5d5qd\" (UID: \"d73f07cc-dce9-4668-968f-14d00c2e1360\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" Jul 01 18:38:55.457853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.457807 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-2nvtf"] Jul 01 18:38:55.460145 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.460120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-2nvtf" Jul 01 18:38:55.462791 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.462770 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 01 18:38:55.462914 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.462770 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-m2wtl\"" Jul 01 18:38:55.463320 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.463303 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 01 18:38:55.479066 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.479046 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-2nvtf"] Jul 01 18:38:55.522548 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.522475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6wkd\" (UniqueName: \"kubernetes.io/projected/d73f07cc-dce9-4668-968f-14d00c2e1360-kube-api-access-h6wkd\") pod \"managed-serviceaccount-addon-agent-8579cbc946-5d5qd\" (UID: \"d73f07cc-dce9-4668-968f-14d00c2e1360\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" Jul 01 18:38:55.522548 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.522521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b960b80f-9641-425c-8e38-ca1d294ceb98-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-4v4xx\" (UID: \"b960b80f-9641-425c-8e38-ca1d294ceb98\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" Jul 01 18:38:55.522733 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.522592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b960b80f-9641-425c-8e38-ca1d294ceb98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-4v4xx\" (UID: \"b960b80f-9641-425c-8e38-ca1d294ceb98\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" Jul 01 18:38:55.522733 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.522669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d73f07cc-dce9-4668-968f-14d00c2e1360-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8579cbc946-5d5qd\" (UID: \"d73f07cc-dce9-4668-968f-14d00c2e1360\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" Jul 01 18:38:55.525067 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.525045 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d73f07cc-dce9-4668-968f-14d00c2e1360-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8579cbc946-5d5qd\" (UID: \"d73f07cc-dce9-4668-968f-14d00c2e1360\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" Jul 01 18:38:55.536964 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.536942 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6wkd\" (UniqueName: \"kubernetes.io/projected/d73f07cc-dce9-4668-968f-14d00c2e1360-kube-api-access-h6wkd\") pod \"managed-serviceaccount-addon-agent-8579cbc946-5d5qd\" (UID: \"d73f07cc-dce9-4668-968f-14d00c2e1360\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" Jul 01 18:38:55.623957 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.623923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b960b80f-9641-425c-8e38-ca1d294ceb98-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-4v4xx\" (UID: \"b960b80f-9641-425c-8e38-ca1d294ceb98\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" Jul 01 18:38:55.624095 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.623972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b960b80f-9641-425c-8e38-ca1d294ceb98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-4v4xx\" (UID: \"b960b80f-9641-425c-8e38-ca1d294ceb98\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" Jul 01 18:38:55.624095 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.624003 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppzf\" (UniqueName: \"kubernetes.io/projected/23ddf9c4-6b4e-4dc1-94fd-6e556b68de76-kube-api-access-wppzf\") pod \"downloads-5499b6fb8f-2nvtf\" (UID: \"23ddf9c4-6b4e-4dc1-94fd-6e556b68de76\") " pod="openshift-console/downloads-5499b6fb8f-2nvtf" Jul 01 18:38:55.624579 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.624542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b960b80f-9641-425c-8e38-ca1d294ceb98-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-4v4xx\" (UID: \"b960b80f-9641-425c-8e38-ca1d294ceb98\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" Jul 01 18:38:55.626266 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.626237 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b960b80f-9641-425c-8e38-ca1d294ceb98-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-4v4xx\" (UID: \"b960b80f-9641-425c-8e38-ca1d294ceb98\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" Jul 01 18:38:55.667550 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.667517 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" Jul 01 18:38:55.724890 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.724854 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppzf\" (UniqueName: \"kubernetes.io/projected/23ddf9c4-6b4e-4dc1-94fd-6e556b68de76-kube-api-access-wppzf\") pod \"downloads-5499b6fb8f-2nvtf\" (UID: \"23ddf9c4-6b4e-4dc1-94fd-6e556b68de76\") " pod="openshift-console/downloads-5499b6fb8f-2nvtf" Jul 01 18:38:55.747947 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.747911 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppzf\" (UniqueName: \"kubernetes.io/projected/23ddf9c4-6b4e-4dc1-94fd-6e556b68de76-kube-api-access-wppzf\") pod \"downloads-5499b6fb8f-2nvtf\" (UID: \"23ddf9c4-6b4e-4dc1-94fd-6e556b68de76\") " pod="openshift-console/downloads-5499b6fb8f-2nvtf" Jul 01 18:38:55.768933 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.768901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-2nvtf" Jul 01 18:38:55.812486 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.812448 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx"] Jul 01 18:38:55.816728 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:55.816699 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb960b80f_9641_425c_8e38_ca1d294ceb98.slice/crio-eb0230dbd5adc60b49b8374f0093aee2fbdfe68ca352ecc66e616e8da8f5b26c WatchSource:0}: Error finding container eb0230dbd5adc60b49b8374f0093aee2fbdfe68ca352ecc66e616e8da8f5b26c: Status 404 returned error can't find the container with id eb0230dbd5adc60b49b8374f0093aee2fbdfe68ca352ecc66e616e8da8f5b26c Jul 01 18:38:55.928582 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:55.928554 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-2nvtf"] Jul 01 18:38:55.931498 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:55.931469 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ddf9c4_6b4e_4dc1_94fd_6e556b68de76.slice/crio-5a092919d24833e7fc3581928800ce39557b355ba5d5a8314bd32f8c6d3cc57b WatchSource:0}: Error finding container 5a092919d24833e7fc3581928800ce39557b355ba5d5a8314bd32f8c6d3cc57b: Status 404 returned error can't find the container with id 5a092919d24833e7fc3581928800ce39557b355ba5d5a8314bd32f8c6d3cc57b Jul 01 18:38:56.344798 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:56.344768 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wz7b8" Jul 01 18:38:56.377853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:56.377814 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" event={"ID":"b960b80f-9641-425c-8e38-ca1d294ceb98","Type":"ContainerStarted","Data":"eb0230dbd5adc60b49b8374f0093aee2fbdfe68ca352ecc66e616e8da8f5b26c"} Jul 01 18:38:56.379090 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:56.379065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-2nvtf" event={"ID":"23ddf9c4-6b4e-4dc1-94fd-6e556b68de76","Type":"ContainerStarted","Data":"5a092919d24833e7fc3581928800ce39557b355ba5d5a8314bd32f8c6d3cc57b"} Jul 01 18:38:56.578994 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:56.578960 2562 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" secret="" err="failed to sync secret cache: timed out waiting for the condition" Jul 01 18:38:56.579158 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:56.579027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" Jul 01 18:38:56.643554 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:56.643521 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vkx98\"" Jul 01 18:38:57.008366 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:57.008334 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd"] Jul 01 18:38:57.012169 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:57.012141 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd73f07cc_dce9_4668_968f_14d00c2e1360.slice/crio-1063a5e6e487fcc467a9ff98b490f211f09e7036c3050d74b9b2d4d731d72f3b WatchSource:0}: Error finding container 1063a5e6e487fcc467a9ff98b490f211f09e7036c3050d74b9b2d4d731d72f3b: Status 404 returned error can't find the container with id 1063a5e6e487fcc467a9ff98b490f211f09e7036c3050d74b9b2d4d731d72f3b Jul 01 18:38:57.384122 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:57.384082 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" event={"ID":"b960b80f-9641-425c-8e38-ca1d294ceb98","Type":"ContainerStarted","Data":"473fbe980f563891edad351232ca554ea4df15d04c4d6f1caff091f4a810c64c"} Jul 01 18:38:57.385261 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:57.385238 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" event={"ID":"d73f07cc-dce9-4668-968f-14d00c2e1360","Type":"ContainerStarted","Data":"1063a5e6e487fcc467a9ff98b490f211f09e7036c3050d74b9b2d4d731d72f3b"} Jul 01 18:38:57.414074 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:57.414013 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-4v4xx" podStartSLOduration=1.355241774 podStartE2EDuration="2.413991661s" totalImagesPullingTime="1.058749887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:57.413218122 +0000 UTC m=+113.928809363" watchObservedRunningTime="2026-07-01 18:38:57.413991661 +0000 UTC m=+113.929582900" Jul 01 18:38:59.054326 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:59.054286 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:59.056935 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:59.056911 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/99cabad8-b09b-48f1-a331-7325be9882c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzrfw\" (UID: \"99cabad8-b09b-48f1-a331-7325be9882c0\") " pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:59.330199 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:59.330073 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rzrfw" Jul 01 18:38:59.597825 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:38:59.597758 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rzrfw"] Jul 01 18:38:59.601526 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:38:59.601496 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cabad8_b09b_48f1_a331_7325be9882c0.slice/crio-392a7161b4db54d0cdee524f727902cc5cc98b6a34135293253640fb8c2f642a WatchSource:0}: Error finding container 392a7161b4db54d0cdee524f727902cc5cc98b6a34135293253640fb8c2f642a: Status 404 returned error can't find the container with id 392a7161b4db54d0cdee524f727902cc5cc98b6a34135293253640fb8c2f642a Jul 01 18:39:00.401824 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:00.401112 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" event={"ID":"d73f07cc-dce9-4668-968f-14d00c2e1360","Type":"ContainerStarted","Data":"3a024ec3447364ba7acc5d299880a899b5bc68ef627322d7ec434457d8bd0821"} Jul 01 18:39:00.404191 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:00.404113 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rzrfw" event={"ID":"99cabad8-b09b-48f1-a331-7325be9882c0","Type":"ContainerStarted","Data":"392a7161b4db54d0cdee524f727902cc5cc98b6a34135293253640fb8c2f642a"} Jul 01 18:39:00.423274 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:00.423143 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8579cbc946-5d5qd" podStartSLOduration=2.92249565 podStartE2EDuration="5.423121072s" totalImagesPullingTime="2.500625422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:00.422305848 +0000 UTC m=+116.937897089" watchObservedRunningTime="2026-07-01 18:39:00.423121072 +0000 UTC m=+116.938712313" Jul 01 18:39:03.117241 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:03.117203 2562 patch_prober.go:28] interesting pod/image-registry-bc86fc8cf-w7hmh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 18:39:03.117708 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:03.117281 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" podUID="90660e9d-a455-4cb2-8bbb-8e66428f94b9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:39:04.397297 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.395591 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fbfbccd6b-2dhq2"] Jul 01 18:39:04.399538 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.399511 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.402933 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.402902 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 01 18:39:04.403290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.403269 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-47s2p\"" Jul 01 18:39:04.403564 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.403545 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 01 18:39:04.404208 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.403797 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 01 18:39:04.404320 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.404285 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 01 18:39:04.404599 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.404566 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 01 18:39:04.409763 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.409736 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 01 18:39:04.413869 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.413835 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fbfbccd6b-2dhq2"] Jul 01 18:39:04.422688 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.422654 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rzrfw" event={"ID":"99cabad8-b09b-48f1-a331-7325be9882c0","Type":"ContainerStarted","Data":"1b9b275d24fc77797fe28d96f94e6b3203f4c0bc54751cff3e2e91d8a0232b6d"} Jul 01 18:39:04.499834 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.499786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-config\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.500018 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.499988 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-trusted-ca-bundle\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.500067 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.500037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-service-ca\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.500098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.500069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-oauth-serving-cert\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.500151 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.500097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-serving-cert\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.500151 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.500120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlkjv\" (UniqueName: \"kubernetes.io/projected/cbaa55d9-76ae-4223-b18b-6f10e157dab2-kube-api-access-hlkjv\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.500266 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.500156 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-oauth-config\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.601311 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.601269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-trusted-ca-bundle\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.601395 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.601313 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-service-ca\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.601395 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.601341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-oauth-serving-cert\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.601395 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.601367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-serving-cert\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.601559 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.601502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlkjv\" (UniqueName: \"kubernetes.io/projected/cbaa55d9-76ae-4223-b18b-6f10e157dab2-kube-api-access-hlkjv\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.601559 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.601543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-oauth-config\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.601670 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.601575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-config\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.602129 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.602105 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-service-ca\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.602239 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.602140 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-oauth-serving-cert\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.602328 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.602302 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-config\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.603040 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.603015 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-trusted-ca-bundle\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.604636 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.604601 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-serving-cert\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.604762 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.604707 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-oauth-config\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.616214 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.616192 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlkjv\" (UniqueName: \"kubernetes.io/projected/cbaa55d9-76ae-4223-b18b-6f10e157dab2-kube-api-access-hlkjv\") pod \"console-5fbfbccd6b-2dhq2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.713603 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.713568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:04.851307 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:04.851274 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fbfbccd6b-2dhq2"] Jul 01 18:39:04.854565 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:39:04.854540 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbaa55d9_76ae_4223_b18b_6f10e157dab2.slice/crio-b329a57f8bac6160065607815eb9592690688425f29843eae7349f412654868f WatchSource:0}: Error finding container b329a57f8bac6160065607815eb9592690688425f29843eae7349f412654868f: Status 404 returned error can't find the container with id b329a57f8bac6160065607815eb9592690688425f29843eae7349f412654868f Jul 01 18:39:05.344264 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:05.344201 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:39:05.431503 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:05.430872 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rzrfw" event={"ID":"99cabad8-b09b-48f1-a331-7325be9882c0","Type":"ContainerStarted","Data":"6ab456b4a35063e409d09e30ffb9827b18e81847f02371db78755bd5a0468070"} Jul 01 18:39:05.432708 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:05.432679 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fbfbccd6b-2dhq2" event={"ID":"cbaa55d9-76ae-4223-b18b-6f10e157dab2","Type":"ContainerStarted","Data":"b329a57f8bac6160065607815eb9592690688425f29843eae7349f412654868f"} Jul 01 18:39:13.492687 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.492653 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6"] Jul 01 18:39:13.497185 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.497154 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.499449 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.499429 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 18:39:13.500198 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.500174 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 01 18:39:13.500297 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.500202 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 01 18:39:13.501239 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.501217 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 18:39:13.501386 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.501368 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 18:39:13.501464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.501417 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7s8fm\"" Jul 01 18:39:13.523164 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.523133 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6"] Jul 01 18:39:13.559386 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.559350 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wmlhj"] Jul 01 18:39:13.563052 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.563030 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-mkf4s"] Jul 01 18:39:13.563181 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.563162 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.565901 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.565880 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 18:39:13.566027 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.565976 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mtwh7\"" Jul 01 18:39:13.566099 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.566038 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 18:39:13.566297 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.566276 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 18:39:13.566412 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.566379 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.568458 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.568438 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qv5k8\"" Jul 01 18:39:13.568458 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.568450 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 01 18:39:13.568458 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.568455 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 01 18:39:13.568859 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.568828 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 01 18:39:13.579615 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.579716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.579716 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579704 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-sys\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.579834 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-textfile\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.579834 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scllv\" (UniqueName: \"kubernetes.io/projected/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-api-access-scllv\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.579949 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579837 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-tls\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.579949 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579887 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-wtmp\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.579949 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a95d3879-6a5f-48a9-b564-704fb4068c32-metrics-client-ca\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.579949 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdc8ae1e-8518-44d7-befb-97872eb358d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.580115 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.579992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-accelerators-collector-config\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.580115 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.580115 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdc8ae1e-8518-44d7-befb-97872eb358d9-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.580242 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580128 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h2vf\" (UniqueName: \"kubernetes.io/projected/cdc8ae1e-8518-44d7-befb-97872eb358d9-kube-api-access-5h2vf\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.580242 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.580242 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba158e2-d820-45d2-b2ad-d49801b0a2af-metrics-client-ca\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.580242 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-root\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.580409 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz8cm\" (UniqueName: \"kubernetes.io/projected/a95d3879-6a5f-48a9-b564-704fb4068c32-kube-api-access-jz8cm\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.580409 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2ba158e2-d820-45d2-b2ad-d49801b0a2af-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.580409 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.580382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cdc8ae1e-8518-44d7-befb-97872eb358d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.590589 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.590570 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-mkf4s"] Jul 01 18:39:13.681687 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681655 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-textfile\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.681905 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scllv\" (UniqueName: \"kubernetes.io/projected/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-api-access-scllv\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.681905 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-tls\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.681905 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-wtmp\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.681905 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a95d3879-6a5f-48a9-b564-704fb4068c32-metrics-client-ca\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.681905 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdc8ae1e-8518-44d7-befb-97872eb358d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.681905 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-accelerators-collector-config\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.681905 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdc8ae1e-8518-44d7-befb-97872eb358d9-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h2vf\" (UniqueName: \"kubernetes.io/projected/cdc8ae1e-8518-44d7-befb-97872eb358d9-kube-api-access-5h2vf\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.681987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba158e2-d820-45d2-b2ad-d49801b0a2af-metrics-client-ca\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-root\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682087 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz8cm\" (UniqueName: \"kubernetes.io/projected/a95d3879-6a5f-48a9-b564-704fb4068c32-kube-api-access-jz8cm\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-textfile\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682149 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2ba158e2-d820-45d2-b2ad-d49801b0a2af-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682139 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-root\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cdc8ae1e-8518-44d7-befb-97872eb358d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.682290 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682295 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.682881 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682331 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-sys\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.682881 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682416 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-sys\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.682881 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682421 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-wtmp\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.682881 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682662 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2ba158e2-d820-45d2-b2ad-d49801b0a2af-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.682881 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdc8ae1e-8518-44d7-befb-97872eb358d9-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.683251 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.682914 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba158e2-d820-45d2-b2ad-d49801b0a2af-metrics-client-ca\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.683940 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.683855 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-accelerators-collector-config\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.684094 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.684044 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a95d3879-6a5f-48a9-b564-704fb4068c32-metrics-client-ca\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.684822 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.684446 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.685185 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.685120 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.685621 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.685581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cdc8ae1e-8518-44d7-befb-97872eb358d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.685748 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.685726 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdc8ae1e-8518-44d7-befb-97872eb358d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.686108 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.686060 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.686453 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.686418 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.686693 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.686675 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a95d3879-6a5f-48a9-b564-704fb4068c32-node-exporter-tls\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.694173 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.694150 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h2vf\" (UniqueName: \"kubernetes.io/projected/cdc8ae1e-8518-44d7-befb-97872eb358d9-kube-api-access-5h2vf\") pod \"openshift-state-metrics-57f99c8b49-4xpv6\" (UID: \"cdc8ae1e-8518-44d7-befb-97872eb358d9\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.694350 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.694326 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scllv\" (UniqueName: \"kubernetes.io/projected/2ba158e2-d820-45d2-b2ad-d49801b0a2af-kube-api-access-scllv\") pod \"kube-state-metrics-845b954455-mkf4s\" (UID: \"2ba158e2-d820-45d2-b2ad-d49801b0a2af\") " pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.696316 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.696281 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz8cm\" (UniqueName: \"kubernetes.io/projected/a95d3879-6a5f-48a9-b564-704fb4068c32-kube-api-access-jz8cm\") pod \"node-exporter-wmlhj\" (UID: \"a95d3879-6a5f-48a9-b564-704fb4068c32\") " pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.782909 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.782811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:39:13.785443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.785420 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973-metrics-certs\") pod \"network-metrics-daemon-npbg7\" (UID: \"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973\") " pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:39:13.807562 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.807528 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" Jul 01 18:39:13.875804 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.875762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wmlhj" Jul 01 18:39:13.878021 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.877994 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2vz9\"" Jul 01 18:39:13.882043 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.882015 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" Jul 01 18:39:13.885700 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:13.885680 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npbg7" Jul 01 18:39:14.757933 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.757899 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:39:14.768370 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.768344 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.773663 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.773618 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 01 18:39:14.773795 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.773660 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 01 18:39:14.773795 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.773707 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 01 18:39:14.773795 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.773764 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7tzrs\"" Jul 01 18:39:14.774155 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.774102 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 01 18:39:14.774292 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.774264 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 01 18:39:14.774362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.774347 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 01 18:39:14.774475 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.774449 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 01 18:39:14.774584 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.774507 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 01 18:39:14.774820 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.774800 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 01 18:39:14.790603 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790572 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.790782 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.790782 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-volume\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.790782 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.790782 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-out\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.790782 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.791166 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.791166 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrjlr\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-kube-api-access-nrjlr\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.791166 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.790992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-web-config\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.791166 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.791038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.791166 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.791083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.791166 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.791109 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.791458 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.791182 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.793172 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.793151 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:39:14.891600 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891558 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-out\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.891815 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891605 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.891815 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.891815 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrjlr\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-kube-api-access-nrjlr\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.891815 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-web-config\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.891815 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.891815 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891766 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.891815 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.892205 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.892205 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.892205 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.892205 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.891978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-volume\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.892205 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.892002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.892205 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.892125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.892935 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.892739 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.894111 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.894086 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.894872 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.894791 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-out\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.895240 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.895215 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-web-config\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.895883 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.895828 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.896448 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.896404 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-volume\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.896741 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.896692 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.896741 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.896718 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.897509 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.897485 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.897635 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.897610 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.897769 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.897713 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:14.906429 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:14.906409 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrjlr\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-kube-api-access-nrjlr\") pod \"alertmanager-main-0\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:15.080116 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:15.080041 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:39:16.216082 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.215946 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-npbg7"] Jul 01 18:39:16.217961 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.217933 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:39:16.421227 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.421194 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-mkf4s"] Jul 01 18:39:16.423287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.423262 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6"] Jul 01 18:39:16.425186 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:39:16.425151 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba158e2_d820_45d2_b2ad_d49801b0a2af.slice/crio-f0adf27acffe749e9300bf0f4aef5ba75b7987fa2c858f294963f3a050db350b WatchSource:0}: Error finding container f0adf27acffe749e9300bf0f4aef5ba75b7987fa2c858f294963f3a050db350b: Status 404 returned error can't find the container with id f0adf27acffe749e9300bf0f4aef5ba75b7987fa2c858f294963f3a050db350b Jul 01 18:39:16.426718 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:39:16.426689 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc8ae1e_8518_44d7_befb_97872eb358d9.slice/crio-5f4fa702c5f184e3e9c6ce55427183d68fc12817249334858eb84eed50a9901d WatchSource:0}: Error finding container 5f4fa702c5f184e3e9c6ce55427183d68fc12817249334858eb84eed50a9901d: Status 404 returned error can't find the container with id 5f4fa702c5f184e3e9c6ce55427183d68fc12817249334858eb84eed50a9901d Jul 01 18:39:16.476544 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.476497 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerStarted","Data":"88dfa55bdc99b4272f92bf699b46c132634ec3a612deb235f9e794f4284313f3"} Jul 01 18:39:16.479059 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.479002 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" event={"ID":"cdc8ae1e-8518-44d7-befb-97872eb358d9","Type":"ContainerStarted","Data":"5f4fa702c5f184e3e9c6ce55427183d68fc12817249334858eb84eed50a9901d"} Jul 01 18:39:16.481874 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.481621 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-2nvtf" event={"ID":"23ddf9c4-6b4e-4dc1-94fd-6e556b68de76","Type":"ContainerStarted","Data":"c62d4bacfbb810dc9d817504992f64ce1e7dbdbd5fdcf8dfdbc4062bcc159b21"} Jul 01 18:39:16.482308 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.482290 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-2nvtf" Jul 01 18:39:16.486186 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.486126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rzrfw" event={"ID":"99cabad8-b09b-48f1-a331-7325be9882c0","Type":"ContainerStarted","Data":"4104e6a8650e9924e67e69c3f4d0d070ff580dbcc6eeaf4faaf0837769e7e748"} Jul 01 18:39:16.490207 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.490183 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wmlhj" event={"ID":"a95d3879-6a5f-48a9-b564-704fb4068c32","Type":"ContainerStarted","Data":"fe27b7912b5551acbb524fab40a46498abcbefe5ab2faceb7a307f1c9bee3e0e"} Jul 01 18:39:16.492034 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.492011 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npbg7" event={"ID":"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973","Type":"ContainerStarted","Data":"aa4bb50ea188c0fbbd9783b39816c733337b98593461eb30f56deb20ed47ba1d"} Jul 01 18:39:16.494654 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.494631 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-2nvtf" Jul 01 18:39:16.495237 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.495196 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fbfbccd6b-2dhq2" event={"ID":"cbaa55d9-76ae-4223-b18b-6f10e157dab2","Type":"ContainerStarted","Data":"6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0"} Jul 01 18:39:16.496898 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.496870 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" event={"ID":"2ba158e2-d820-45d2-b2ad-d49801b0a2af","Type":"ContainerStarted","Data":"f0adf27acffe749e9300bf0f4aef5ba75b7987fa2c858f294963f3a050db350b"} Jul 01 18:39:16.542541 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.541412 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-2nvtf" podStartSLOduration=1.4463995490000001 podStartE2EDuration="21.541388206s" totalImagesPullingTime="20.094988657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:16.507414795 +0000 UTC m=+133.023006035" watchObservedRunningTime="2026-07-01 18:39:16.541388206 +0000 UTC m=+133.056979470" Jul 01 18:39:16.578650 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.578231 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rzrfw" podStartSLOduration=33.367252015 podStartE2EDuration="49.578206969s" totalImagesPullingTime="16.210954954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:38:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:16.542714018 +0000 UTC m=+133.058305258" watchObservedRunningTime="2026-07-01 18:39:16.578206969 +0000 UTC m=+133.093798217" Jul 01 18:39:16.579403 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:16.579354 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fbfbccd6b-2dhq2" podStartSLOduration=1.464551551 podStartE2EDuration="12.579335738s" totalImagesPullingTime="11.114784187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:16.577321992 +0000 UTC m=+133.092913234" watchObservedRunningTime="2026-07-01 18:39:16.579335738 +0000 UTC m=+133.094926977" Jul 01 18:39:17.299076 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:17.298127 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bc86fc8cf-w7hmh"] Jul 01 18:39:17.506517 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:17.506041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" event={"ID":"cdc8ae1e-8518-44d7-befb-97872eb358d9","Type":"ContainerStarted","Data":"ed156d9c9360130903b30ae66283a97347135d2bb70b9b125160cba48578cf45"} Jul 01 18:39:17.506517 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:17.506102 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" event={"ID":"cdc8ae1e-8518-44d7-befb-97872eb358d9","Type":"ContainerStarted","Data":"65dfb0485daae321ac77e8b36c34e97cb0943ee1b54d312d5b63fd3f17a6f5a6"} Jul 01 18:39:19.643273 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:19.643240 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fbfbccd6b-2dhq2"] Jul 01 18:39:20.136627 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.135966 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:39:20.148731 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.148699 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.159411 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.159382 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v825x\"" Jul 01 18:39:20.172276 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.172149 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 18:39:20.172553 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.172529 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 18:39:20.172553 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.172512 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 18:39:20.172707 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.172606 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 18:39:20.175231 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.175125 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 18:39:20.177945 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.177713 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 18:39:20.187965 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.187928 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 18:39:20.194884 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.194836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 18:39:20.196192 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.195907 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 18:39:20.198665 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.197872 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 18:39:20.198665 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.198025 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 18:39:20.198665 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.198112 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 18:39:20.198665 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.198323 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5te2lqsp3kq1f\"" Jul 01 18:39:20.198932 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.198834 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 18:39:20.207803 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.207765 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 18:39:20.233613 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.233398 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 18:39:20.233897 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.233768 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:39:20.247002 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.246973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247133 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247133 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247071 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247133 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247295 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247158 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247295 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247295 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247295 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247268 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247375 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247399 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247422 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247493 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9jdp\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-kube-api-access-v9jdp\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247813 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247813 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247563 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247813 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247813 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.247813 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.247727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.348588 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348312 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.348588 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.348588 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.348588 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348754 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9jdp\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-kube-api-access-v9jdp\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.348969 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349072 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.349362 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.350332 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.350332 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.350332 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349365 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.350332 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349373 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.350332 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.349449 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.350673 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.350367 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.351414 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.351360 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.354599 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.352203 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.354599 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.354564 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.355252 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.355229 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.357873 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.356388 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.357873 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.357010 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.357873 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.357777 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.358501 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.358371 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.360238 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.360029 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config-out\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.360390 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.360323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.360390 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.360364 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.360562 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.360383 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.360990 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.360966 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.362464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.362442 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.362762 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.362743 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-web-config\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.379990 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.379960 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9jdp\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-kube-api-access-v9jdp\") pod \"prometheus-k8s-0\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.465258 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.465059 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:20.528955 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.527428 2562 generic.go:356] "Generic (PLEG): container finished" podID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerID="716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e" exitCode=0 Jul 01 18:39:20.528955 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.527550 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerDied","Data":"716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e"} Jul 01 18:39:20.533142 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.533010 2562 generic.go:356] "Generic (PLEG): container finished" podID="a95d3879-6a5f-48a9-b564-704fb4068c32" containerID="c1d9e9fd15109b79bf58ae7fbcd726933b7bb0fc14ecd8466a154e581b2ef7d4" exitCode=0 Jul 01 18:39:20.533142 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.533100 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wmlhj" event={"ID":"a95d3879-6a5f-48a9-b564-704fb4068c32","Type":"ContainerDied","Data":"c1d9e9fd15109b79bf58ae7fbcd726933b7bb0fc14ecd8466a154e581b2ef7d4"} Jul 01 18:39:20.535990 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.535837 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npbg7" event={"ID":"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973","Type":"ContainerStarted","Data":"5ec3352471c1cd5bb5ed01b0b11e6b4bbc8e5e8a1a9a0dee885a19140afc4e6a"} Jul 01 18:39:20.538751 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.538693 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" event={"ID":"2ba158e2-d820-45d2-b2ad-d49801b0a2af","Type":"ContainerStarted","Data":"42aa7db2ff27608d1e1069d1038d09637e5821e1ea2bccf5aa8ef0fdf5e8f368"} Jul 01 18:39:20.538894 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.538785 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" event={"ID":"2ba158e2-d820-45d2-b2ad-d49801b0a2af","Type":"ContainerStarted","Data":"24a85cf83a66ab90112aaa10371d790b71d592a30ba2d49e2c4666014163c0ea"} Jul 01 18:39:20.792861 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:20.792816 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:39:20.795683 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:39:20.795655 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d8ee70e_91df_4e63_acf8_34ee4892ad86.slice/crio-94561b582f1d112634988b0855b91e4136668dee0571a95c70c3a172f5c1ce5f WatchSource:0}: Error finding container 94561b582f1d112634988b0855b91e4136668dee0571a95c70c3a172f5c1ce5f: Status 404 returned error can't find the container with id 94561b582f1d112634988b0855b91e4136668dee0571a95c70c3a172f5c1ce5f Jul 01 18:39:21.549900 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.549837 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" exitCode=0 Jul 01 18:39:21.550134 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.549946 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerDied","Data":"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2"} Jul 01 18:39:21.550134 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.550030 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerStarted","Data":"94561b582f1d112634988b0855b91e4136668dee0571a95c70c3a172f5c1ce5f"} Jul 01 18:39:21.552606 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.552577 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" event={"ID":"cdc8ae1e-8518-44d7-befb-97872eb358d9","Type":"ContainerStarted","Data":"986ee85634b7a4c6f41c6c8ba7d3e9aa7a67f3db0a78a1d07f1eabff83cb0c3f"} Jul 01 18:39:21.555214 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.555193 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wmlhj" event={"ID":"a95d3879-6a5f-48a9-b564-704fb4068c32","Type":"ContainerStarted","Data":"5a949bb0c033b77d4f45835265137a7c0d8d9bb36062a00bff2a392852858484"} Jul 01 18:39:21.555328 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.555236 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wmlhj" event={"ID":"a95d3879-6a5f-48a9-b564-704fb4068c32","Type":"ContainerStarted","Data":"9b50c85571eac4465625e6ab8194840d9603bf4eeff0dacbc3470523a7284cd6"} Jul 01 18:39:21.557189 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.557163 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npbg7" event={"ID":"d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973","Type":"ContainerStarted","Data":"45667013465ca354b5a9b30c7e390a337cf508a37f1661328d21dffa2d2cadca"} Jul 01 18:39:21.559748 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.559726 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" event={"ID":"2ba158e2-d820-45d2-b2ad-d49801b0a2af","Type":"ContainerStarted","Data":"74db90656f602058eabbd5adaac7e3e757149d51a2c45fdfb64ba7c2cc77ec21"} Jul 01 18:39:21.626802 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.626736 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wmlhj" podStartSLOduration=4.556888557 podStartE2EDuration="8.626715172s" totalImagesPullingTime="3.960157137s" totalInitContainerRuntime="109.669478ms" isStatefulPod=true podCreationTimestamp="2026-07-01 18:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:21.626255116 +0000 UTC m=+138.141846356" watchObservedRunningTime="2026-07-01 18:39:21.626715172 +0000 UTC m=+138.142306410" Jul 01 18:39:21.669116 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.669055 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-4xpv6" podStartSLOduration=4.891314568 podStartE2EDuration="8.6690349s" totalImagesPullingTime="3.777720332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:21.667436073 +0000 UTC m=+138.183027313" watchObservedRunningTime="2026-07-01 18:39:21.6690349 +0000 UTC m=+138.184626140" Jul 01 18:39:21.701014 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.700950 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-mkf4s" podStartSLOduration=5.181651444 podStartE2EDuration="8.700926882s" totalImagesPullingTime="3.519275438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:21.699093379 +0000 UTC m=+138.214684618" watchObservedRunningTime="2026-07-01 18:39:21.700926882 +0000 UTC m=+138.216518121" Jul 01 18:39:21.737051 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:21.736998 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-npbg7" podStartSLOduration=134.011607187 podStartE2EDuration="2m17.736975479s" totalImagesPullingTime="3.725368292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:21.735811792 +0000 UTC m=+138.251403032" watchObservedRunningTime="2026-07-01 18:39:21.736975479 +0000 UTC m=+138.252566723" Jul 01 18:39:23.572878 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:23.572825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerStarted","Data":"5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5"} Jul 01 18:39:24.583432 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:24.583381 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerStarted","Data":"ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e"} Jul 01 18:39:24.584033 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:24.583472 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerStarted","Data":"1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5"} Jul 01 18:39:24.584033 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:24.583519 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerStarted","Data":"f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c"} Jul 01 18:39:24.584033 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:24.583558 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerStarted","Data":"19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb"} Jul 01 18:39:24.721446 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:24.721397 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:26.594016 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:26.593938 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerStarted","Data":"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e"} Jul 01 18:39:26.594370 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:26.594034 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerStarted","Data":"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78"} Jul 01 18:39:27.601566 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:27.601531 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerStarted","Data":"336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7"} Jul 01 18:39:27.655514 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:27.655447 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.308002179 podStartE2EDuration="13.655405599s" totalImagesPullingTime="9.321017293s" totalInitContainerRuntime="26.386127ms" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:27.654323383 +0000 UTC m=+144.169914656" watchObservedRunningTime="2026-07-01 18:39:27.655405599 +0000 UTC m=+144.170996836" Jul 01 18:39:28.608234 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:28.608200 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerStarted","Data":"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436"} Jul 01 18:39:28.608551 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:28.608293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerStarted","Data":"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203"} Jul 01 18:39:29.615369 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:29.615335 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerStarted","Data":"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913"} Jul 01 18:39:29.615875 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:29.615398 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerStarted","Data":"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6"} Jul 01 18:39:29.658913 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:29.658863 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.122389426 podStartE2EDuration="9.658829939s" totalImagesPullingTime="6.517503172s" totalInitContainerRuntime="18.937341ms" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:29.656735974 +0000 UTC m=+146.172327214" watchObservedRunningTime="2026-07-01 18:39:29.658829939 +0000 UTC m=+146.174421175" Jul 01 18:39:30.466355 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:30.466315 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:39:37.930107 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:37.930049 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-59bd7_87032a95-9729-4094-a901-5ac3827cc3e1/serve-healthcheck-canary/0.log" Jul 01 18:39:42.330151 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.330092 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" podUID="90660e9d-a455-4cb2-8bbb-8e66428f94b9" containerName="registry" containerID="cri-o://961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd" gracePeriod=30 Jul 01 18:39:42.604245 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.604217 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:39:42.668978 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.668943 2562 generic.go:356] "Generic (PLEG): container finished" podID="90660e9d-a455-4cb2-8bbb-8e66428f94b9" containerID="961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd" exitCode=0 Jul 01 18:39:42.669146 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.668987 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" Jul 01 18:39:42.669146 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.669016 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" event={"ID":"90660e9d-a455-4cb2-8bbb-8e66428f94b9","Type":"ContainerDied","Data":"961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd"} Jul 01 18:39:42.669146 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.669059 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc86fc8cf-w7hmh" event={"ID":"90660e9d-a455-4cb2-8bbb-8e66428f94b9","Type":"ContainerDied","Data":"d353398c84b7053d93d2dec83f57f1f8e579739db98c776e11c87ea2c924702a"} Jul 01 18:39:42.669146 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.669079 2562 scope.go:117] "RemoveContainer" containerID="961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd" Jul 01 18:39:42.676942 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.676916 2562 scope.go:117] "RemoveContainer" containerID="961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd" Jul 01 18:39:42.677197 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:39:42.677177 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd\": container with ID starting with 961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd not found: ID does not exist" containerID="961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd" Jul 01 18:39:42.677252 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.677206 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd"} err="failed to get container status \"961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd\": rpc error: code = NotFound desc = could not find container \"961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd\": container with ID starting with 961a0dabcace0b77225f9169e2e1a4f2d620cbaafaf1c68cbd9a252c75c659fd not found: ID does not exist" Jul 01 18:39:42.759114 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759074 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48k7s\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-kube-api-access-48k7s\") pod \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " Jul 01 18:39:42.759114 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759121 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-installation-pull-secrets\") pod \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " Jul 01 18:39:42.759334 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759150 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-trusted-ca\") pod \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " Jul 01 18:39:42.759334 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759172 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") pod \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " Jul 01 18:39:42.759334 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759206 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-image-registry-private-configuration\") pod \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " Jul 01 18:39:42.759334 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759272 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-certificates\") pod \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " Jul 01 18:39:42.759334 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759302 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-bound-sa-token\") pod \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " Jul 01 18:39:42.759581 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759345 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90660e9d-a455-4cb2-8bbb-8e66428f94b9-ca-trust-extracted\") pod \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\" (UID: \"90660e9d-a455-4cb2-8bbb-8e66428f94b9\") " Jul 01 18:39:42.759659 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759614 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "90660e9d-a455-4cb2-8bbb-8e66428f94b9" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:39:42.759774 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.759734 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "90660e9d-a455-4cb2-8bbb-8e66428f94b9" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:39:42.762101 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.762068 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "90660e9d-a455-4cb2-8bbb-8e66428f94b9" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:39:42.762254 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.762220 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "90660e9d-a455-4cb2-8bbb-8e66428f94b9" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:39:42.762331 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.762256 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "90660e9d-a455-4cb2-8bbb-8e66428f94b9" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:39:42.762385 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.762330 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-kube-api-access-48k7s" (OuterVolumeSpecName: "kube-api-access-48k7s") pod "90660e9d-a455-4cb2-8bbb-8e66428f94b9" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9"). InnerVolumeSpecName "kube-api-access-48k7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:39:42.762385 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.762338 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "90660e9d-a455-4cb2-8bbb-8e66428f94b9" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:39:42.767975 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.767952 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90660e9d-a455-4cb2-8bbb-8e66428f94b9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "90660e9d-a455-4cb2-8bbb-8e66428f94b9" (UID: "90660e9d-a455-4cb2-8bbb-8e66428f94b9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:39:42.860930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.860820 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90660e9d-a455-4cb2-8bbb-8e66428f94b9-ca-trust-extracted\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:42.860930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.860874 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48k7s\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-kube-api-access-48k7s\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:42.860930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.860884 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-installation-pull-secrets\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:42.860930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.860893 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-trusted-ca\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:42.860930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.860902 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-tls\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:42.860930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.860913 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90660e9d-a455-4cb2-8bbb-8e66428f94b9-image-registry-private-configuration\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:42.860930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.860923 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90660e9d-a455-4cb2-8bbb-8e66428f94b9-registry-certificates\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:42.860930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:42.860933 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90660e9d-a455-4cb2-8bbb-8e66428f94b9-bound-sa-token\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:43.006150 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:43.006109 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bc86fc8cf-w7hmh"] Jul 01 18:39:43.010797 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:43.010769 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-bc86fc8cf-w7hmh"] Jul 01 18:39:43.961957 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:43.961922 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90660e9d-a455-4cb2-8bbb-8e66428f94b9" path="/var/lib/kubelet/pods/90660e9d-a455-4cb2-8bbb-8e66428f94b9/volumes" Jul 01 18:39:44.671499 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:44.671418 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5fbfbccd6b-2dhq2" podUID="cbaa55d9-76ae-4223-b18b-6f10e157dab2" containerName="console" containerID="cri-o://6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0" gracePeriod=15 Jul 01 18:39:44.964483 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:44.964446 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:45.078912 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.078874 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-service-ca\") pod \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " Jul 01 18:39:45.079081 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.078968 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlkjv\" (UniqueName: \"kubernetes.io/projected/cbaa55d9-76ae-4223-b18b-6f10e157dab2-kube-api-access-hlkjv\") pod \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " Jul 01 18:39:45.079081 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.078989 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-serving-cert\") pod \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " Jul 01 18:39:45.079081 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079013 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-oauth-serving-cert\") pod \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " Jul 01 18:39:45.079081 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079040 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-oauth-config\") pod \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " Jul 01 18:39:45.079081 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079057 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-config\") pod \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " Jul 01 18:39:45.079323 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079156 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-trusted-ca-bundle\") pod \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\" (UID: \"cbaa55d9-76ae-4223-b18b-6f10e157dab2\") " Jul 01 18:39:45.079409 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079372 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-service-ca" (OuterVolumeSpecName: "service-ca") pod "cbaa55d9-76ae-4223-b18b-6f10e157dab2" (UID: "cbaa55d9-76ae-4223-b18b-6f10e157dab2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:39:45.079491 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079475 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-service-ca\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:45.079561 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079513 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-config" (OuterVolumeSpecName: "console-config") pod "cbaa55d9-76ae-4223-b18b-6f10e157dab2" (UID: "cbaa55d9-76ae-4223-b18b-6f10e157dab2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:39:45.079561 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079525 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cbaa55d9-76ae-4223-b18b-6f10e157dab2" (UID: "cbaa55d9-76ae-4223-b18b-6f10e157dab2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:39:45.079780 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.079758 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cbaa55d9-76ae-4223-b18b-6f10e157dab2" (UID: "cbaa55d9-76ae-4223-b18b-6f10e157dab2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:39:45.081366 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.081345 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cbaa55d9-76ae-4223-b18b-6f10e157dab2" (UID: "cbaa55d9-76ae-4223-b18b-6f10e157dab2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:39:45.081452 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.081391 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cbaa55d9-76ae-4223-b18b-6f10e157dab2" (UID: "cbaa55d9-76ae-4223-b18b-6f10e157dab2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:39:45.081452 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.081444 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbaa55d9-76ae-4223-b18b-6f10e157dab2-kube-api-access-hlkjv" (OuterVolumeSpecName: "kube-api-access-hlkjv") pod "cbaa55d9-76ae-4223-b18b-6f10e157dab2" (UID: "cbaa55d9-76ae-4223-b18b-6f10e157dab2"). InnerVolumeSpecName "kube-api-access-hlkjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:39:45.180279 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.180243 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlkjv\" (UniqueName: \"kubernetes.io/projected/cbaa55d9-76ae-4223-b18b-6f10e157dab2-kube-api-access-hlkjv\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:45.180279 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.180271 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-serving-cert\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:45.180279 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.180282 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-oauth-serving-cert\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:45.180279 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.180290 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-oauth-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:45.180536 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.180300 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-console-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:45.180536 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.180308 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbaa55d9-76ae-4223-b18b-6f10e157dab2-trusted-ca-bundle\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:39:45.684387 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.684349 2562 generic.go:356] "Generic (PLEG): container finished" podID="cbaa55d9-76ae-4223-b18b-6f10e157dab2" containerID="6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0" exitCode=0 Jul 01 18:39:45.684387 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.684398 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fbfbccd6b-2dhq2" Jul 01 18:39:45.684614 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.684432 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fbfbccd6b-2dhq2" event={"ID":"cbaa55d9-76ae-4223-b18b-6f10e157dab2","Type":"ContainerDied","Data":"6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0"} Jul 01 18:39:45.684614 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.684511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fbfbccd6b-2dhq2" event={"ID":"cbaa55d9-76ae-4223-b18b-6f10e157dab2","Type":"ContainerDied","Data":"b329a57f8bac6160065607815eb9592690688425f29843eae7349f412654868f"} Jul 01 18:39:45.684614 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.684532 2562 scope.go:117] "RemoveContainer" containerID="6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0" Jul 01 18:39:45.693024 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.693000 2562 scope.go:117] "RemoveContainer" containerID="6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0" Jul 01 18:39:45.693285 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:39:45.693262 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0\": container with ID starting with 6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0 not found: ID does not exist" containerID="6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0" Jul 01 18:39:45.693358 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.693292 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0"} err="failed to get container status \"6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0\": rpc error: code = NotFound desc = could not find container \"6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0\": container with ID starting with 6e85dea336c308f6a34fa27b6f0d5928008a3a4df244d32db6e889afd6e1e5a0 not found: ID does not exist" Jul 01 18:39:45.711316 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.711287 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fbfbccd6b-2dhq2"] Jul 01 18:39:45.717759 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.717732 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fbfbccd6b-2dhq2"] Jul 01 18:39:45.961654 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:39:45.961567 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbaa55d9-76ae-4223-b18b-6f10e157dab2" path="/var/lib/kubelet/pods/cbaa55d9-76ae-4223-b18b-6f10e157dab2/volumes" Jul 01 18:40:20.466527 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:20.466487 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:20.485763 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:20.485734 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:20.846592 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:20.846511 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:34.337990 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.337953 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:40:34.338529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.338234 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="alertmanager" containerID="cri-o://5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5" gracePeriod=120 Jul 01 18:40:34.338529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.338310 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy" containerID="cri-o://1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5" gracePeriod=120 Jul 01 18:40:34.338529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.338331 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy-web" containerID="cri-o://f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c" gracePeriod=120 Jul 01 18:40:34.338529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.338355 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="prom-label-proxy" containerID="cri-o://336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7" gracePeriod=120 Jul 01 18:40:34.338529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.338394 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="config-reloader" containerID="cri-o://19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb" gracePeriod=120 Jul 01 18:40:34.338529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.338309 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy-metric" containerID="cri-o://ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e" gracePeriod=120 Jul 01 18:40:34.886071 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886037 2562 generic.go:356] "Generic (PLEG): container finished" podID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerID="336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7" exitCode=0 Jul 01 18:40:34.886071 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886062 2562 generic.go:356] "Generic (PLEG): container finished" podID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerID="ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e" exitCode=0 Jul 01 18:40:34.886071 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886070 2562 generic.go:356] "Generic (PLEG): container finished" podID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerID="1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5" exitCode=0 Jul 01 18:40:34.886071 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886075 2562 generic.go:356] "Generic (PLEG): container finished" podID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerID="19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb" exitCode=0 Jul 01 18:40:34.886071 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886081 2562 generic.go:356] "Generic (PLEG): container finished" podID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerID="5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5" exitCode=0 Jul 01 18:40:34.886440 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886111 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerDied","Data":"336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7"} Jul 01 18:40:34.886440 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886221 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerDied","Data":"ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e"} Jul 01 18:40:34.886440 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886259 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerDied","Data":"1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5"} Jul 01 18:40:34.886440 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886302 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerDied","Data":"19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb"} Jul 01 18:40:34.886440 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:34.886340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerDied","Data":"5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5"} Jul 01 18:40:35.580610 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.580583 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:35.721535 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721438 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-main-db\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.721535 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721492 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.721755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721544 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-volume\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.721755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721581 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.721755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721616 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrjlr\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-kube-api-access-nrjlr\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.721755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721639 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-web-config\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.721755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721672 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-web\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.721755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721698 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-out\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.721755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721730 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-tls-assets\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.722132 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721782 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-trusted-ca-bundle\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.722132 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721812 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-main-tls\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.722132 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721874 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-metrics-client-ca\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.722132 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721938 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-cluster-tls-config\") pod \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\" (UID: \"19d0274e-6222-4b32-82df-72ad7e3fd4f6\") " Jul 01 18:40:35.723450 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.721803 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:40:35.723450 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.723390 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:35.723669 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.723495 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:35.724610 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.724581 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-out" (OuterVolumeSpecName: "config-out") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:40:35.725072 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.725034 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:35.725367 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.725339 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:35.725448 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.725364 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:35.725448 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.725354 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-kube-api-access-nrjlr" (OuterVolumeSpecName: "kube-api-access-nrjlr") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "kube-api-access-nrjlr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:40:35.725798 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.725768 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-volume" (OuterVolumeSpecName: "config-volume") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:35.725933 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.725794 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:35.726000 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.725975 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:40:35.728819 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.728794 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:35.735644 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.735621 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-web-config" (OuterVolumeSpecName: "web-config") pod "19d0274e-6222-4b32-82df-72ad7e3fd4f6" (UID: "19d0274e-6222-4b32-82df-72ad7e3fd4f6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:35.823201 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823156 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-main-db\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823201 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823200 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823201 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823211 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-volume\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823223 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823233 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrjlr\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-kube-api-access-nrjlr\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823241 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-web-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823251 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823258 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19d0274e-6222-4b32-82df-72ad7e3fd4f6-config-out\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823265 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19d0274e-6222-4b32-82df-72ad7e3fd4f6-tls-assets\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823274 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823282 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-secret-alertmanager-main-tls\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823290 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d0274e-6222-4b32-82df-72ad7e3fd4f6-metrics-client-ca\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.823443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.823299 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19d0274e-6222-4b32-82df-72ad7e3fd4f6-cluster-tls-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:35.892925 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.892892 2562 generic.go:356] "Generic (PLEG): container finished" podID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerID="f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c" exitCode=0 Jul 01 18:40:35.893082 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.892966 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:35.893082 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.892979 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerDied","Data":"f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c"} Jul 01 18:40:35.893082 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.893049 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19d0274e-6222-4b32-82df-72ad7e3fd4f6","Type":"ContainerDied","Data":"88dfa55bdc99b4272f92bf699b46c132634ec3a612deb235f9e794f4284313f3"} Jul 01 18:40:35.893082 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.893083 2562 scope.go:117] "RemoveContainer" containerID="336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7" Jul 01 18:40:35.900645 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.900628 2562 scope.go:117] "RemoveContainer" containerID="ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e" Jul 01 18:40:35.907663 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.907637 2562 scope.go:117] "RemoveContainer" containerID="1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5" Jul 01 18:40:35.914293 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.914277 2562 scope.go:117] "RemoveContainer" containerID="f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c" Jul 01 18:40:35.921762 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.921746 2562 scope.go:117] "RemoveContainer" containerID="19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb" Jul 01 18:40:35.922197 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.922178 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:40:35.925372 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.925352 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:40:35.929615 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.929599 2562 scope.go:117] "RemoveContainer" containerID="5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5" Jul 01 18:40:35.936477 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.936456 2562 scope.go:117] "RemoveContainer" containerID="716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e" Jul 01 18:40:35.943204 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.943189 2562 scope.go:117] "RemoveContainer" containerID="336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7" Jul 01 18:40:35.943449 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:35.943419 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7\": container with ID starting with 336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7 not found: ID does not exist" containerID="336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7" Jul 01 18:40:35.943497 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.943457 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7"} err="failed to get container status \"336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7\": rpc error: code = NotFound desc = could not find container \"336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7\": container with ID starting with 336b4f10b083f181167a58aeab910173c8b84b0192ef0d150614ae986d6efdc7 not found: ID does not exist" Jul 01 18:40:35.943497 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.943474 2562 scope.go:117] "RemoveContainer" containerID="ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e" Jul 01 18:40:35.943683 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:35.943668 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e\": container with ID starting with ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e not found: ID does not exist" containerID="ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e" Jul 01 18:40:35.943732 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.943687 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e"} err="failed to get container status \"ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e\": rpc error: code = NotFound desc = could not find container \"ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e\": container with ID starting with ddd6db1c3c9554d6875580e47167749c1e014da4f93371e015900df956dd9e1e not found: ID does not exist" Jul 01 18:40:35.943732 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.943700 2562 scope.go:117] "RemoveContainer" containerID="1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5" Jul 01 18:40:35.943951 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:35.943931 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5\": container with ID starting with 1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5 not found: ID does not exist" containerID="1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5" Jul 01 18:40:35.943993 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.943957 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5"} err="failed to get container status \"1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5\": rpc error: code = NotFound desc = could not find container \"1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5\": container with ID starting with 1da220a0410a3a2025175c0ea0f3317500be00d81025d000311ae6d0b2ff93a5 not found: ID does not exist" Jul 01 18:40:35.943993 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.943973 2562 scope.go:117] "RemoveContainer" containerID="f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c" Jul 01 18:40:35.944196 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:35.944180 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c\": container with ID starting with f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c not found: ID does not exist" containerID="f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c" Jul 01 18:40:35.944233 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.944199 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c"} err="failed to get container status \"f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c\": rpc error: code = NotFound desc = could not find container \"f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c\": container with ID starting with f0610ef2546cc861c533bc4abf0269432758d26e352a7595848c91c26257966c not found: ID does not exist" Jul 01 18:40:35.944233 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.944212 2562 scope.go:117] "RemoveContainer" containerID="19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb" Jul 01 18:40:35.944416 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:35.944403 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb\": container with ID starting with 19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb not found: ID does not exist" containerID="19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb" Jul 01 18:40:35.944458 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.944418 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb"} err="failed to get container status \"19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb\": rpc error: code = NotFound desc = could not find container \"19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb\": container with ID starting with 19cd8c1ba615ad9627f1c13d569a98f74a12b408dc01512096b4201dba76becb not found: ID does not exist" Jul 01 18:40:35.944458 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.944429 2562 scope.go:117] "RemoveContainer" containerID="5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5" Jul 01 18:40:35.944598 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:35.944584 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5\": container with ID starting with 5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5 not found: ID does not exist" containerID="5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5" Jul 01 18:40:35.944638 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.944600 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5"} err="failed to get container status \"5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5\": rpc error: code = NotFound desc = could not find container \"5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5\": container with ID starting with 5abc26fc033f7a4adc449d778d09bb6579fbcd92962d32d331cf1e3dda6926b5 not found: ID does not exist" Jul 01 18:40:35.944638 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.944610 2562 scope.go:117] "RemoveContainer" containerID="716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e" Jul 01 18:40:35.944815 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:35.944800 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e\": container with ID starting with 716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e not found: ID does not exist" containerID="716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e" Jul 01 18:40:35.944868 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.944819 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e"} err="failed to get container status \"716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e\": rpc error: code = NotFound desc = could not find container \"716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e\": container with ID starting with 716b3fe2e3700b3bd72ba54c763396f2904411dcc98bb99a45f772f9cee8bd1e not found: ID does not exist" Jul 01 18:40:35.960577 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.960554 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" path="/var/lib/kubelet/pods/19d0274e-6222-4b32-82df-72ad7e3fd4f6/volumes" Jul 01 18:40:35.963646 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.963623 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:40:35.963952 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.963937 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90660e9d-a455-4cb2-8bbb-8e66428f94b9" containerName="registry" Jul 01 18:40:35.964018 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.963955 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="90660e9d-a455-4cb2-8bbb-8e66428f94b9" containerName="registry" Jul 01 18:40:35.964018 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.963968 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="alertmanager" Jul 01 18:40:35.964018 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.963976 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="alertmanager" Jul 01 18:40:35.964018 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.963990 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy-metric" Jul 01 18:40:35.964018 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964000 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy-metric" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964021 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="init-config-reloader" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964030 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="init-config-reloader" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964048 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbaa55d9-76ae-4223-b18b-6f10e157dab2" containerName="console" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964057 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbaa55d9-76ae-4223-b18b-6f10e157dab2" containerName="console" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964074 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="config-reloader" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964082 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="config-reloader" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964093 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy-web" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964103 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy-web" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964112 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964121 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964130 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="prom-label-proxy" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964138 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="prom-label-proxy" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964204 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="alertmanager" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964215 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="config-reloader" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964226 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="90660e9d-a455-4cb2-8bbb-8e66428f94b9" containerName="registry" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964236 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbaa55d9-76ae-4223-b18b-6f10e157dab2" containerName="console" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964246 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy-metric" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964254 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy" Jul 01 18:40:35.964263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964263 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="kube-rbac-proxy-web" Jul 01 18:40:35.965130 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.964274 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d0274e-6222-4b32-82df-72ad7e3fd4f6" containerName="prom-label-proxy" Jul 01 18:40:35.968497 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.968479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:35.971047 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.971032 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 01 18:40:35.971730 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.971684 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 01 18:40:35.971966 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.971951 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 01 18:40:35.972071 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.972051 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7tzrs\"" Jul 01 18:40:35.972140 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.972070 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 01 18:40:35.972208 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.972191 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 01 18:40:35.972287 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.972271 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 01 18:40:35.972516 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.972498 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 01 18:40:35.972636 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.972619 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 01 18:40:35.976536 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.976517 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 01 18:40:35.987357 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:35.987336 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:40:36.126138 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126343 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126151 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b068d9-24e4-4f8f-94d6-609b5e26004f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126343 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126343 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/25b068d9-24e4-4f8f-94d6-609b5e26004f-config-out\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126343 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126230 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126343 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/25b068d9-24e4-4f8f-94d6-609b5e26004f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126343 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126343 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126331 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25b068d9-24e4-4f8f-94d6-609b5e26004f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-config-volume\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-web-config\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126470 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jjsc\" (UniqueName: \"kubernetes.io/projected/25b068d9-24e4-4f8f-94d6-609b5e26004f-kube-api-access-2jjsc\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.126628 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.126567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/25b068d9-24e4-4f8f-94d6-609b5e26004f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.227915 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.227814 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/25b068d9-24e4-4f8f-94d6-609b5e26004f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.227915 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.227889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228142 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.227923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b068d9-24e4-4f8f-94d6-609b5e26004f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228142 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.227955 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228142 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.227981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/25b068d9-24e4-4f8f-94d6-609b5e26004f-config-out\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228142 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.228004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228142 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.228125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/25b068d9-24e4-4f8f-94d6-609b5e26004f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228381 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.228183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228381 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.228209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228381 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.228236 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25b068d9-24e4-4f8f-94d6-609b5e26004f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228381 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.228280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-config-volume\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228381 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.228311 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-web-config\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.228381 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.228342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jjsc\" (UniqueName: \"kubernetes.io/projected/25b068d9-24e4-4f8f-94d6-609b5e26004f-kube-api-access-2jjsc\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.229062 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.229025 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b068d9-24e4-4f8f-94d6-609b5e26004f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.229329 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.229302 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/25b068d9-24e4-4f8f-94d6-609b5e26004f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.230001 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.229953 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25b068d9-24e4-4f8f-94d6-609b5e26004f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.231309 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.231283 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/25b068d9-24e4-4f8f-94d6-609b5e26004f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.231416 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.231317 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.231481 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.231461 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.231588 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.231561 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.231762 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.231737 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.231762 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.231756 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.231915 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.231900 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/25b068d9-24e4-4f8f-94d6-609b5e26004f-config-out\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.232036 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.232017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-config-volume\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.232667 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.232651 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/25b068d9-24e4-4f8f-94d6-609b5e26004f-web-config\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.247555 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.247533 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jjsc\" (UniqueName: \"kubernetes.io/projected/25b068d9-24e4-4f8f-94d6-609b5e26004f-kube-api-access-2jjsc\") pod \"alertmanager-main-0\" (UID: \"25b068d9-24e4-4f8f-94d6-609b5e26004f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.277499 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.277467 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 18:40:36.419536 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.419512 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 18:40:36.421925 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:40:36.421899 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b068d9_24e4_4f8f_94d6_609b5e26004f.slice/crio-d425012863049d3e5cbb504223c279915cd2dc96492f6de14afd9c2c1ed0c6b4 WatchSource:0}: Error finding container d425012863049d3e5cbb504223c279915cd2dc96492f6de14afd9c2c1ed0c6b4: Status 404 returned error can't find the container with id d425012863049d3e5cbb504223c279915cd2dc96492f6de14afd9c2c1ed0c6b4 Jul 01 18:40:36.899030 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.898996 2562 generic.go:356] "Generic (PLEG): container finished" podID="25b068d9-24e4-4f8f-94d6-609b5e26004f" containerID="6144bdf5b4ea0d5e6663165731c7dd1605ae1689d09b4159750f4723c2ac26c9" exitCode=0 Jul 01 18:40:36.899476 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.899049 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"25b068d9-24e4-4f8f-94d6-609b5e26004f","Type":"ContainerDied","Data":"6144bdf5b4ea0d5e6663165731c7dd1605ae1689d09b4159750f4723c2ac26c9"} Jul 01 18:40:36.899476 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:36.899123 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"25b068d9-24e4-4f8f-94d6-609b5e26004f","Type":"ContainerStarted","Data":"d425012863049d3e5cbb504223c279915cd2dc96492f6de14afd9c2c1ed0c6b4"} Jul 01 18:40:37.908264 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:37.908222 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"25b068d9-24e4-4f8f-94d6-609b5e26004f","Type":"ContainerStarted","Data":"5b540ac10b651ef49b7b12a7af8fc710ca095cc2afdb0015ec4fc41dc459dd32"} Jul 01 18:40:37.908645 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:37.908291 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"25b068d9-24e4-4f8f-94d6-609b5e26004f","Type":"ContainerStarted","Data":"3dbfe2e8bab7262f181f95824e26f3090768308155635108faa21716b8e0aa07"} Jul 01 18:40:37.908645 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:37.908317 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"25b068d9-24e4-4f8f-94d6-609b5e26004f","Type":"ContainerStarted","Data":"63d71d41eb6f0e89a430ee811f200b6385652d054bcf63a56da201b35d41a595"} Jul 01 18:40:37.908645 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:37.908347 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"25b068d9-24e4-4f8f-94d6-609b5e26004f","Type":"ContainerStarted","Data":"0e2b9adfea0accbb665dc2522b85b90544bb05ae738eb8efb42624723f730cb1"} Jul 01 18:40:37.908645 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:37.908370 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"25b068d9-24e4-4f8f-94d6-609b5e26004f","Type":"ContainerStarted","Data":"af19ddefa49039de4e13d552ddbf12d89425c3008f55dcd4211e19765d8e6393"} Jul 01 18:40:37.908645 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:37.908392 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"25b068d9-24e4-4f8f-94d6-609b5e26004f","Type":"ContainerStarted","Data":"a059c06792a192047627babd8c65aeabe9b2f1c8ce7c5685a46b7a4fa668148e"} Jul 01 18:40:37.939900 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:37.939825 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.9262428910000002 podStartE2EDuration="2.939807401s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.56451ms" isStatefulPod=false podCreationTimestamp="2026-07-01 18:40:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:40:37.938683426 +0000 UTC m=+214.454274690" watchObservedRunningTime="2026-07-01 18:40:37.939807401 +0000 UTC m=+214.455398640" Jul 01 18:40:38.610010 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.609976 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:40:38.610282 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.610253 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="prometheus" containerID="cri-o://6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" gracePeriod=600 Jul 01 18:40:38.610353 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.610276 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy" containerID="cri-o://821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" gracePeriod=600 Jul 01 18:40:38.610353 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.610315 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="config-reloader" containerID="cri-o://404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" gracePeriod=600 Jul 01 18:40:38.610461 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.610339 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy-thanos" containerID="cri-o://befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" gracePeriod=600 Jul 01 18:40:38.610461 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.610286 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="thanos-sidecar" containerID="cri-o://08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" gracePeriod=600 Jul 01 18:40:38.610461 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.610305 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy-web" containerID="cri-o://392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" gracePeriod=600 Jul 01 18:40:38.861670 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.861612 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:38.917051 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917017 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" exitCode=0 Jul 01 18:40:38.917051 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917046 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" exitCode=0 Jul 01 18:40:38.917051 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917055 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" exitCode=0 Jul 01 18:40:38.917051 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917061 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" exitCode=0 Jul 01 18:40:38.917051 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917067 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" exitCode=0 Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917075 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" exitCode=0 Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerDied","Data":"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913"} Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917104 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917173 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerDied","Data":"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6"} Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerDied","Data":"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436"} Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917214 2562 scope.go:117] "RemoveContainer" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917238 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerDied","Data":"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203"} Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917264 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerDied","Data":"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e"} Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917295 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerDied","Data":"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78"} Jul 01 18:40:38.917784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.917320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2d8ee70e-91df-4e63-acf8-34ee4892ad86","Type":"ContainerDied","Data":"94561b582f1d112634988b0855b91e4136668dee0571a95c70c3a172f5c1ce5f"} Jul 01 18:40:38.925168 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.925149 2562 scope.go:117] "RemoveContainer" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" Jul 01 18:40:38.932376 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.932352 2562 scope.go:117] "RemoveContainer" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" Jul 01 18:40:38.939448 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.939431 2562 scope.go:117] "RemoveContainer" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" Jul 01 18:40:38.947048 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.947026 2562 scope.go:117] "RemoveContainer" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" Jul 01 18:40:38.953805 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.953787 2562 scope.go:117] "RemoveContainer" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" Jul 01 18:40:38.956039 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956017 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-db\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956153 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956054 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-grpc-tls\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956153 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956088 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-web-config\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956153 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956117 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-kubelet-serving-ca-bundle\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956153 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956145 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956453 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956430 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956520 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956485 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9jdp\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-kube-api-access-v9jdp\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956579 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956535 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-1\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956579 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956559 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:38.956679 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956568 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-thanos-prometheus-http-client-file\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956679 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956623 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-trusted-ca-bundle\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956679 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956650 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-0\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956679 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956674 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-tls-assets\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956896 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956721 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-2\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956896 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956774 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-metrics-client-ca\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956896 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956800 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956896 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956832 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-metrics-client-certs\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.956896 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956877 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-kube-rbac-proxy\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.957150 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956903 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config-out\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.957150 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956933 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-serving-certs-ca-bundle\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.957150 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.956967 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-tls\") pod \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\" (UID: \"2d8ee70e-91df-4e63-acf8-34ee4892ad86\") " Jul 01 18:40:38.957898 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.957332 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:38.957898 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.957378 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:40:38.958046 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.958027 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:38.958222 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.958196 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:38.959079 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.959052 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:38.959520 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.959409 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:38.959520 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.959481 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.960469 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.960441 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:38.961434 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.961406 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:38.962458 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.962416 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.962717 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.962688 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.962955 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.962829 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.963030 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.962945 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.963030 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.962981 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.963328 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.963275 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.963534 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.963496 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config-out" (OuterVolumeSpecName: "config-out") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:40:38.963669 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.963588 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config" (OuterVolumeSpecName: "config") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.964065 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.964042 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-kube-api-access-v9jdp" (OuterVolumeSpecName: "kube-api-access-v9jdp") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "kube-api-access-v9jdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:40:38.964756 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.964738 2562 scope.go:117] "RemoveContainer" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" Jul 01 18:40:38.964999 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.964974 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:40:38.971587 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.971555 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-web-config" (OuterVolumeSpecName: "web-config") pod "2d8ee70e-91df-4e63-acf8-34ee4892ad86" (UID: "2d8ee70e-91df-4e63-acf8-34ee4892ad86"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:38.971995 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.971978 2562 scope.go:117] "RemoveContainer" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" Jul 01 18:40:38.972305 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:38.972280 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": container with ID starting with befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913 not found: ID does not exist" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" Jul 01 18:40:38.972375 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.972312 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913"} err="failed to get container status \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": rpc error: code = NotFound desc = could not find container \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": container with ID starting with befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913 not found: ID does not exist" Jul 01 18:40:38.972375 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.972330 2562 scope.go:117] "RemoveContainer" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" Jul 01 18:40:38.972590 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:38.972572 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": container with ID starting with 821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6 not found: ID does not exist" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" Jul 01 18:40:38.972658 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.972598 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6"} err="failed to get container status \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": rpc error: code = NotFound desc = could not find container \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": container with ID starting with 821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6 not found: ID does not exist" Jul 01 18:40:38.972658 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.972623 2562 scope.go:117] "RemoveContainer" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" Jul 01 18:40:38.972914 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:38.972894 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": container with ID starting with 392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436 not found: ID does not exist" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" Jul 01 18:40:38.972965 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.972918 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436"} err="failed to get container status \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": rpc error: code = NotFound desc = could not find container \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": container with ID starting with 392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436 not found: ID does not exist" Jul 01 18:40:38.972965 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.972932 2562 scope.go:117] "RemoveContainer" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" Jul 01 18:40:38.973155 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:38.973139 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": container with ID starting with 08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203 not found: ID does not exist" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" Jul 01 18:40:38.973198 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.973160 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203"} err="failed to get container status \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": rpc error: code = NotFound desc = could not find container \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": container with ID starting with 08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203 not found: ID does not exist" Jul 01 18:40:38.973198 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.973173 2562 scope.go:117] "RemoveContainer" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" Jul 01 18:40:38.973395 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:38.973378 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": container with ID starting with 404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e not found: ID does not exist" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" Jul 01 18:40:38.973450 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.973401 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e"} err="failed to get container status \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": rpc error: code = NotFound desc = could not find container \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": container with ID starting with 404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e not found: ID does not exist" Jul 01 18:40:38.973450 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.973420 2562 scope.go:117] "RemoveContainer" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" Jul 01 18:40:38.973637 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:38.973620 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": container with ID starting with 6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78 not found: ID does not exist" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" Jul 01 18:40:38.973675 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.973642 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78"} err="failed to get container status \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": rpc error: code = NotFound desc = could not find container \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": container with ID starting with 6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78 not found: ID does not exist" Jul 01 18:40:38.973675 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.973654 2562 scope.go:117] "RemoveContainer" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" Jul 01 18:40:38.973874 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:40:38.973834 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": container with ID starting with 33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2 not found: ID does not exist" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" Jul 01 18:40:38.973915 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.973882 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2"} err="failed to get container status \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": rpc error: code = NotFound desc = could not find container \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": container with ID starting with 33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2 not found: ID does not exist" Jul 01 18:40:38.973915 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.973901 2562 scope.go:117] "RemoveContainer" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" Jul 01 18:40:38.974136 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974119 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913"} err="failed to get container status \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": rpc error: code = NotFound desc = could not find container \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": container with ID starting with befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913 not found: ID does not exist" Jul 01 18:40:38.974202 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974138 2562 scope.go:117] "RemoveContainer" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" Jul 01 18:40:38.974341 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974325 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6"} err="failed to get container status \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": rpc error: code = NotFound desc = could not find container \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": container with ID starting with 821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6 not found: ID does not exist" Jul 01 18:40:38.974398 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974343 2562 scope.go:117] "RemoveContainer" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" Jul 01 18:40:38.974543 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974525 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436"} err="failed to get container status \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": rpc error: code = NotFound desc = could not find container \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": container with ID starting with 392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436 not found: ID does not exist" Jul 01 18:40:38.974590 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974544 2562 scope.go:117] "RemoveContainer" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" Jul 01 18:40:38.974763 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974739 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203"} err="failed to get container status \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": rpc error: code = NotFound desc = could not find container \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": container with ID starting with 08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203 not found: ID does not exist" Jul 01 18:40:38.974763 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974763 2562 scope.go:117] "RemoveContainer" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" Jul 01 18:40:38.974958 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974936 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e"} err="failed to get container status \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": rpc error: code = NotFound desc = could not find container \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": container with ID starting with 404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e not found: ID does not exist" Jul 01 18:40:38.975022 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.974961 2562 scope.go:117] "RemoveContainer" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" Jul 01 18:40:38.975182 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.975164 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78"} err="failed to get container status \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": rpc error: code = NotFound desc = could not find container \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": container with ID starting with 6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78 not found: ID does not exist" Jul 01 18:40:38.975249 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.975183 2562 scope.go:117] "RemoveContainer" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" Jul 01 18:40:38.975411 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.975387 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2"} err="failed to get container status \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": rpc error: code = NotFound desc = could not find container \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": container with ID starting with 33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2 not found: ID does not exist" Jul 01 18:40:38.975411 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.975405 2562 scope.go:117] "RemoveContainer" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" Jul 01 18:40:38.975629 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.975612 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913"} err="failed to get container status \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": rpc error: code = NotFound desc = could not find container \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": container with ID starting with befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913 not found: ID does not exist" Jul 01 18:40:38.975682 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.975630 2562 scope.go:117] "RemoveContainer" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" Jul 01 18:40:38.975821 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.975803 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6"} err="failed to get container status \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": rpc error: code = NotFound desc = could not find container \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": container with ID starting with 821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6 not found: ID does not exist" Jul 01 18:40:38.975893 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.975822 2562 scope.go:117] "RemoveContainer" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" Jul 01 18:40:38.976082 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976063 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436"} err="failed to get container status \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": rpc error: code = NotFound desc = could not find container \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": container with ID starting with 392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436 not found: ID does not exist" Jul 01 18:40:38.976128 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976082 2562 scope.go:117] "RemoveContainer" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" Jul 01 18:40:38.976289 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976274 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203"} err="failed to get container status \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": rpc error: code = NotFound desc = could not find container \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": container with ID starting with 08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203 not found: ID does not exist" Jul 01 18:40:38.976334 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976289 2562 scope.go:117] "RemoveContainer" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" Jul 01 18:40:38.976490 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976471 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e"} err="failed to get container status \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": rpc error: code = NotFound desc = could not find container \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": container with ID starting with 404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e not found: ID does not exist" Jul 01 18:40:38.976528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976491 2562 scope.go:117] "RemoveContainer" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" Jul 01 18:40:38.976686 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976672 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78"} err="failed to get container status \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": rpc error: code = NotFound desc = could not find container \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": container with ID starting with 6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78 not found: ID does not exist" Jul 01 18:40:38.976725 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976685 2562 scope.go:117] "RemoveContainer" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" Jul 01 18:40:38.976904 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976885 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2"} err="failed to get container status \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": rpc error: code = NotFound desc = could not find container \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": container with ID starting with 33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2 not found: ID does not exist" Jul 01 18:40:38.976953 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.976907 2562 scope.go:117] "RemoveContainer" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" Jul 01 18:40:38.977102 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977086 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913"} err="failed to get container status \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": rpc error: code = NotFound desc = could not find container \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": container with ID starting with befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913 not found: ID does not exist" Jul 01 18:40:38.977102 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977101 2562 scope.go:117] "RemoveContainer" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" Jul 01 18:40:38.977281 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977265 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6"} err="failed to get container status \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": rpc error: code = NotFound desc = could not find container \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": container with ID starting with 821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6 not found: ID does not exist" Jul 01 18:40:38.977322 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977281 2562 scope.go:117] "RemoveContainer" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" Jul 01 18:40:38.977499 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977481 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436"} err="failed to get container status \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": rpc error: code = NotFound desc = could not find container \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": container with ID starting with 392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436 not found: ID does not exist" Jul 01 18:40:38.977499 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977498 2562 scope.go:117] "RemoveContainer" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" Jul 01 18:40:38.977669 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977654 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203"} err="failed to get container status \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": rpc error: code = NotFound desc = could not find container \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": container with ID starting with 08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203 not found: ID does not exist" Jul 01 18:40:38.977707 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977669 2562 scope.go:117] "RemoveContainer" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" Jul 01 18:40:38.977883 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977866 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e"} err="failed to get container status \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": rpc error: code = NotFound desc = could not find container \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": container with ID starting with 404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e not found: ID does not exist" Jul 01 18:40:38.977930 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.977883 2562 scope.go:117] "RemoveContainer" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" Jul 01 18:40:38.978078 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978063 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78"} err="failed to get container status \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": rpc error: code = NotFound desc = could not find container \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": container with ID starting with 6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78 not found: ID does not exist" Jul 01 18:40:38.978115 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978079 2562 scope.go:117] "RemoveContainer" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" Jul 01 18:40:38.978265 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978252 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2"} err="failed to get container status \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": rpc error: code = NotFound desc = could not find container \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": container with ID starting with 33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2 not found: ID does not exist" Jul 01 18:40:38.978304 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978265 2562 scope.go:117] "RemoveContainer" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" Jul 01 18:40:38.978440 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978425 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913"} err="failed to get container status \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": rpc error: code = NotFound desc = could not find container \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": container with ID starting with befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913 not found: ID does not exist" Jul 01 18:40:38.978478 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978440 2562 scope.go:117] "RemoveContainer" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" Jul 01 18:40:38.978609 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978595 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6"} err="failed to get container status \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": rpc error: code = NotFound desc = could not find container \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": container with ID starting with 821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6 not found: ID does not exist" Jul 01 18:40:38.978646 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978610 2562 scope.go:117] "RemoveContainer" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" Jul 01 18:40:38.978795 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978780 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436"} err="failed to get container status \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": rpc error: code = NotFound desc = could not find container \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": container with ID starting with 392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436 not found: ID does not exist" Jul 01 18:40:38.978831 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978795 2562 scope.go:117] "RemoveContainer" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" Jul 01 18:40:38.979015 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.978999 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203"} err="failed to get container status \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": rpc error: code = NotFound desc = could not find container \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": container with ID starting with 08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203 not found: ID does not exist" Jul 01 18:40:38.979062 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979016 2562 scope.go:117] "RemoveContainer" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" Jul 01 18:40:38.979220 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979203 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e"} err="failed to get container status \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": rpc error: code = NotFound desc = could not find container \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": container with ID starting with 404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e not found: ID does not exist" Jul 01 18:40:38.979220 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979219 2562 scope.go:117] "RemoveContainer" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" Jul 01 18:40:38.979389 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979373 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78"} err="failed to get container status \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": rpc error: code = NotFound desc = could not find container \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": container with ID starting with 6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78 not found: ID does not exist" Jul 01 18:40:38.979431 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979390 2562 scope.go:117] "RemoveContainer" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" Jul 01 18:40:38.979553 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979537 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2"} err="failed to get container status \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": rpc error: code = NotFound desc = could not find container \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": container with ID starting with 33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2 not found: ID does not exist" Jul 01 18:40:38.979604 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979553 2562 scope.go:117] "RemoveContainer" containerID="befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913" Jul 01 18:40:38.979711 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979695 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913"} err="failed to get container status \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": rpc error: code = NotFound desc = could not find container \"befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913\": container with ID starting with befde840779b0b09a17f311b394f1c634976d1407846a49328e230db536c3913 not found: ID does not exist" Jul 01 18:40:38.979711 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979710 2562 scope.go:117] "RemoveContainer" containerID="821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6" Jul 01 18:40:38.979897 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979876 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6"} err="failed to get container status \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": rpc error: code = NotFound desc = could not find container \"821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6\": container with ID starting with 821219275ea2aaf8a7959f397948c1b16afd0d6f38fc349244e5934815e839d6 not found: ID does not exist" Jul 01 18:40:38.979947 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.979903 2562 scope.go:117] "RemoveContainer" containerID="392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436" Jul 01 18:40:38.980115 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980099 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436"} err="failed to get container status \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": rpc error: code = NotFound desc = could not find container \"392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436\": container with ID starting with 392f21b235d9633c1e118f7d7a763c48388310fc9c4758da80c15f2421434436 not found: ID does not exist" Jul 01 18:40:38.980156 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980116 2562 scope.go:117] "RemoveContainer" containerID="08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203" Jul 01 18:40:38.980313 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980295 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203"} err="failed to get container status \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": rpc error: code = NotFound desc = could not find container \"08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203\": container with ID starting with 08525b1e24bef8b00cfd3d23fb2036392d300bcb3dcb30c0c652f09de2888203 not found: ID does not exist" Jul 01 18:40:38.980313 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980312 2562 scope.go:117] "RemoveContainer" containerID="404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e" Jul 01 18:40:38.980475 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980457 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e"} err="failed to get container status \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": rpc error: code = NotFound desc = could not find container \"404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e\": container with ID starting with 404de61deaef7ef5e16d5e56d3a02c6b2b521cf2099faed4b8c43ac21980ba9e not found: ID does not exist" Jul 01 18:40:38.980515 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980479 2562 scope.go:117] "RemoveContainer" containerID="6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78" Jul 01 18:40:38.980637 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980623 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78"} err="failed to get container status \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": rpc error: code = NotFound desc = could not find container \"6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78\": container with ID starting with 6f503251ee36d4dc2fa1a3fd097302fbaf22aad6090b555f7599b5e4d0b19b78 not found: ID does not exist" Jul 01 18:40:38.980673 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980637 2562 scope.go:117] "RemoveContainer" containerID="33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2" Jul 01 18:40:38.980800 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:38.980786 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2"} err="failed to get container status \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": rpc error: code = NotFound desc = could not find container \"33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2\": container with ID starting with 33773e36c7a65bd93d77562c30e9732c337571d31a67802814790ad5b9db94a2 not found: ID does not exist" Jul 01 18:40:39.058109 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058073 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-kube-rbac-proxy\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058109 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058102 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config-out\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058109 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058112 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058121 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058130 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-db\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058138 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-grpc-tls\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058147 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-web-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058155 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058163 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058172 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9jdp\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-kube-api-access-v9jdp\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058181 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058189 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058198 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058206 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058214 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d8ee70e-91df-4e63-acf8-34ee4892ad86-tls-assets\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058223 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058231 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d8ee70e-91df-4e63-acf8-34ee4892ad86-configmap-metrics-client-ca\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058240 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.058314 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.058248 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2d8ee70e-91df-4e63-acf8-34ee4892ad86-secret-metrics-client-certs\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:40:39.246288 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.246244 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:40:39.251944 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.251916 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:40:39.282799 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.282768 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:40:39.283085 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283073 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="config-reloader" Jul 01 18:40:39.283123 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283087 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="config-reloader" Jul 01 18:40:39.283123 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283098 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy-web" Jul 01 18:40:39.283123 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283103 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy-web" Jul 01 18:40:39.283123 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283111 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="init-config-reloader" Jul 01 18:40:39.283123 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283116 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="init-config-reloader" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283128 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="thanos-sidecar" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283133 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="thanos-sidecar" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283140 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="prometheus" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283145 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="prometheus" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283159 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy-thanos" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283164 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy-thanos" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283172 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283177 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283217 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="config-reloader" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283223 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283230 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy-web" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283236 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="prometheus" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283242 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="thanos-sidecar" Jul 01 18:40:39.283272 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.283248 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" containerName="kube-rbac-proxy-thanos" Jul 01 18:40:39.287383 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.287364 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.289970 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.289950 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 18:40:39.290232 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290212 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 18:40:39.290296 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290221 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 18:40:39.290296 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290255 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v825x\"" Jul 01 18:40:39.290296 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290221 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 18:40:39.290464 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290438 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 18:40:39.290545 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290530 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5te2lqsp3kq1f\"" Jul 01 18:40:39.290698 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290682 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 18:40:39.290757 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290726 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 18:40:39.290757 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290740 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 18:40:39.290757 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.290749 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 18:40:39.291284 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.291269 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 18:40:39.291437 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.291422 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 18:40:39.292023 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.292005 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 18:40:39.292117 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.292007 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 18:40:39.293734 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.293714 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 18:40:39.298170 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.298147 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 18:40:39.304837 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.304817 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:40:39.360740 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360740 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360973 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360973 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360973 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360973 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360973 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360835 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360973 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360973 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360900 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.360973 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360920 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg8qp\" (UniqueName: \"kubernetes.io/projected/ff98045f-c140-4839-be98-991560af691a-kube-api-access-lg8qp\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.360989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361029 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361096 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-config\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361123 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff98045f-c140-4839-be98-991560af691a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361173 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361228 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff98045f-c140-4839-be98-991560af691a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.361310 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.361264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.461873 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.461809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.461873 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.461876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.461895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.461913 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.461943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg8qp\" (UniqueName: \"kubernetes.io/projected/ff98045f-c140-4839-be98-991560af691a-kube-api-access-lg8qp\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.461979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462037 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-config\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462098 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff98045f-c140-4839-be98-991560af691a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462129 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff98045f-c140-4839-be98-991560af691a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462262 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462292 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462318 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.462529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.462404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.463036 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.463016 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.463089 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.463067 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.463645 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.463615 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.465929 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.465113 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.465929 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.465158 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.465929 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.465275 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.465929 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.465273 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.465929 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.465507 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.465929 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.465619 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.465929 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.465650 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.465929 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.465824 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.466376 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.466163 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff98045f-c140-4839-be98-991560af691a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.466497 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.466476 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.466557 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.466508 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff98045f-c140-4839-be98-991560af691a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.466680 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.466657 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.467448 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.467427 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.467811 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.467793 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff98045f-c140-4839-be98-991560af691a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.467885 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.467814 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-config\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.468173 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.468157 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff98045f-c140-4839-be98-991560af691a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.472684 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.472667 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg8qp\" (UniqueName: \"kubernetes.io/projected/ff98045f-c140-4839-be98-991560af691a-kube-api-access-lg8qp\") pod \"prometheus-k8s-0\" (UID: \"ff98045f-c140-4839-be98-991560af691a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.597419 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.597320 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:40:39.738441 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.738400 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 18:40:39.742252 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:40:39.742223 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff98045f_c140_4839_be98_991560af691a.slice/crio-0214e88e56f04b2858801b005cc41cc95c142789d345cc7bb224c163f9040d27 WatchSource:0}: Error finding container 0214e88e56f04b2858801b005cc41cc95c142789d345cc7bb224c163f9040d27: Status 404 returned error can't find the container with id 0214e88e56f04b2858801b005cc41cc95c142789d345cc7bb224c163f9040d27 Jul 01 18:40:39.922444 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.922408 2562 generic.go:356] "Generic (PLEG): container finished" podID="ff98045f-c140-4839-be98-991560af691a" containerID="eb62b979728f7d47b57684c870068999b7e2f17cb5e261338080a8b354d2fb34" exitCode=0 Jul 01 18:40:39.922928 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.922488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff98045f-c140-4839-be98-991560af691a","Type":"ContainerDied","Data":"eb62b979728f7d47b57684c870068999b7e2f17cb5e261338080a8b354d2fb34"} Jul 01 18:40:39.922928 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.922553 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff98045f-c140-4839-be98-991560af691a","Type":"ContainerStarted","Data":"0214e88e56f04b2858801b005cc41cc95c142789d345cc7bb224c163f9040d27"} Jul 01 18:40:39.961430 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:39.961401 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d8ee70e-91df-4e63-acf8-34ee4892ad86" path="/var/lib/kubelet/pods/2d8ee70e-91df-4e63-acf8-34ee4892ad86/volumes" Jul 01 18:40:40.930546 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:40.930463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff98045f-c140-4839-be98-991560af691a","Type":"ContainerStarted","Data":"2cf740b491ef679fd02345186a3b770d63378abb71f4fc28361d5a8f8e510379"} Jul 01 18:40:40.930942 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:40.930555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff98045f-c140-4839-be98-991560af691a","Type":"ContainerStarted","Data":"b5f7d01eccd2369b7f6c17627d1c5842079cceefd4cc7945b9aae43f55f1106f"} Jul 01 18:40:40.930942 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:40.930583 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff98045f-c140-4839-be98-991560af691a","Type":"ContainerStarted","Data":"fd0d9f0bfeb4f8ff7bad359f3b6b0b521c2806dac0dee714f982d47df260cccd"} Jul 01 18:40:40.930942 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:40.930607 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff98045f-c140-4839-be98-991560af691a","Type":"ContainerStarted","Data":"3f2b27f86bc4c2a35648fd7e78bbda0a1cb5eadf401665c6b87492fa3fed363e"} Jul 01 18:40:40.930942 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:40.930629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff98045f-c140-4839-be98-991560af691a","Type":"ContainerStarted","Data":"72eadbb3597844543a055a1237b743fb41502c4b5abf70cf4d0ac74a3979ecb3"} Jul 01 18:40:40.930942 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:40.930653 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ff98045f-c140-4839-be98-991560af691a","Type":"ContainerStarted","Data":"f6531fd40e072cb1e66f6fe1d03737a9d5b5cd3b4c1ad3dbe2ccb9b487fc85de"} Jul 01 18:40:40.968489 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:40.968404 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.948504926 podStartE2EDuration="1.968381421s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.876495ms" isStatefulPod=false podCreationTimestamp="2026-07-01 18:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:40:40.966439908 +0000 UTC m=+217.482031165" watchObservedRunningTime="2026-07-01 18:40:40.968381421 +0000 UTC m=+217.483972661" Jul 01 18:40:44.597969 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:40:44.597931 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:41:39.597591 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:41:39.597505 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:41:39.613180 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:41:39.613153 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:41:40.174781 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:41:40.174750 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 18:42:03.884569 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:42:03.884538 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:42:03.887659 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:42:03.887633 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:42:03.889415 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:42:03.889387 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:42:03.896660 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:42:03.896633 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:42:03.900619 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:42:03.900602 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 18:45:12.750686 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.750649 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-kjdth"] Jul 01 18:45:12.753136 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.753119 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-kjdth" Jul 01 18:45:12.758623 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.758599 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-m2nzb\"" Jul 01 18:45:12.758761 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.758670 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 01 18:45:12.758974 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.758957 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 01 18:45:12.759516 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.759309 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 01 18:45:12.764193 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.764168 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-kjdth"] Jul 01 18:45:12.825024 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.824991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbj6h\" (UniqueName: \"kubernetes.io/projected/47a16951-1e0f-4a78-8e9c-e55dc89848e2-kube-api-access-mbj6h\") pod \"s3-init-kjdth\" (UID: \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\") " pod="kserve/s3-init-kjdth" Jul 01 18:45:12.825189 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.825037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47a16951-1e0f-4a78-8e9c-e55dc89848e2-model-cache\") pod \"s3-init-kjdth\" (UID: \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\") " pod="kserve/s3-init-kjdth" Jul 01 18:45:12.926157 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.926122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbj6h\" (UniqueName: \"kubernetes.io/projected/47a16951-1e0f-4a78-8e9c-e55dc89848e2-kube-api-access-mbj6h\") pod \"s3-init-kjdth\" (UID: \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\") " pod="kserve/s3-init-kjdth" Jul 01 18:45:12.926321 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.926181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47a16951-1e0f-4a78-8e9c-e55dc89848e2-model-cache\") pod \"s3-init-kjdth\" (UID: \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\") " pod="kserve/s3-init-kjdth" Jul 01 18:45:12.926549 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.926534 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47a16951-1e0f-4a78-8e9c-e55dc89848e2-model-cache\") pod \"s3-init-kjdth\" (UID: \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\") " pod="kserve/s3-init-kjdth" Jul 01 18:45:12.936115 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:12.936089 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbj6h\" (UniqueName: \"kubernetes.io/projected/47a16951-1e0f-4a78-8e9c-e55dc89848e2-kube-api-access-mbj6h\") pod \"s3-init-kjdth\" (UID: \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\") " pod="kserve/s3-init-kjdth" Jul 01 18:45:13.061804 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:13.061723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-kjdth" Jul 01 18:45:13.195904 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:13.195872 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-kjdth"] Jul 01 18:45:13.201185 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:45:13.201156 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47a16951_1e0f_4a78_8e9c_e55dc89848e2.slice/crio-aae6d2d256c418654ce4740f1e34b1dedc7c460b928bccfcf2f20954310d12ad WatchSource:0}: Error finding container aae6d2d256c418654ce4740f1e34b1dedc7c460b928bccfcf2f20954310d12ad: Status 404 returned error can't find the container with id aae6d2d256c418654ce4740f1e34b1dedc7c460b928bccfcf2f20954310d12ad Jul 01 18:45:13.203038 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:13.203017 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 18:45:13.969546 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:13.969507 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kjdth" event={"ID":"47a16951-1e0f-4a78-8e9c-e55dc89848e2","Type":"ContainerStarted","Data":"aae6d2d256c418654ce4740f1e34b1dedc7c460b928bccfcf2f20954310d12ad"} Jul 01 18:45:17.987637 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:17.987590 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kjdth" event={"ID":"47a16951-1e0f-4a78-8e9c-e55dc89848e2","Type":"ContainerStarted","Data":"84f82da27eb88ef17fa06c90907d6e4082e1dbf60e646616d4139ccd51086a67"} Jul 01 18:45:19.997304 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:19.997275 2562 generic.go:356] "Generic (PLEG): container finished" podID="47a16951-1e0f-4a78-8e9c-e55dc89848e2" containerID="84f82da27eb88ef17fa06c90907d6e4082e1dbf60e646616d4139ccd51086a67" exitCode=0 Jul 01 18:45:19.997785 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:19.997347 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kjdth" event={"ID":"47a16951-1e0f-4a78-8e9c-e55dc89848e2","Type":"ContainerDied","Data":"84f82da27eb88ef17fa06c90907d6e4082e1dbf60e646616d4139ccd51086a67"} Jul 01 18:45:25.028074 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:25.027994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kjdth" event={"ID":"47a16951-1e0f-4a78-8e9c-e55dc89848e2","Type":"ContainerStarted","Data":"eac59e80a70dfbfcfc0d5ef02bea71fa83dcdcf0ebdae517a282b1f9ae1d6de8"} Jul 01 18:45:25.050806 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:45:25.050757 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-kjdth" podStartSLOduration=2.365201899 podStartE2EDuration="13.050741683s" totalImagesPullingTime="9.076041439s" totalInitContainerRuntime="1.609498345s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:45:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:45:25.04832898 +0000 UTC m=+501.563920231" watchObservedRunningTime="2026-07-01 18:45:25.050741683 +0000 UTC m=+501.566332923" Jul 01 18:46:30.281058 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:30.280966 2562 generic.go:356] "Generic (PLEG): container finished" podID="47a16951-1e0f-4a78-8e9c-e55dc89848e2" containerID="eac59e80a70dfbfcfc0d5ef02bea71fa83dcdcf0ebdae517a282b1f9ae1d6de8" exitCode=0 Jul 01 18:46:30.281058 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:30.281035 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kjdth" event={"ID":"47a16951-1e0f-4a78-8e9c-e55dc89848e2","Type":"ContainerDied","Data":"eac59e80a70dfbfcfc0d5ef02bea71fa83dcdcf0ebdae517a282b1f9ae1d6de8"} Jul 01 18:46:31.412781 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:31.412758 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-kjdth" Jul 01 18:46:31.470505 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:31.470468 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbj6h\" (UniqueName: \"kubernetes.io/projected/47a16951-1e0f-4a78-8e9c-e55dc89848e2-kube-api-access-mbj6h\") pod \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\" (UID: \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\") " Jul 01 18:46:31.470668 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:31.470532 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47a16951-1e0f-4a78-8e9c-e55dc89848e2-model-cache\") pod \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\" (UID: \"47a16951-1e0f-4a78-8e9c-e55dc89848e2\") " Jul 01 18:46:31.472701 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:31.472676 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47a16951-1e0f-4a78-8e9c-e55dc89848e2-kube-api-access-mbj6h" (OuterVolumeSpecName: "kube-api-access-mbj6h") pod "47a16951-1e0f-4a78-8e9c-e55dc89848e2" (UID: "47a16951-1e0f-4a78-8e9c-e55dc89848e2"). InnerVolumeSpecName "kube-api-access-mbj6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:46:31.492361 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:31.492299 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47a16951-1e0f-4a78-8e9c-e55dc89848e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "47a16951-1e0f-4a78-8e9c-e55dc89848e2" (UID: "47a16951-1e0f-4a78-8e9c-e55dc89848e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:46:31.571531 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:31.571451 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47a16951-1e0f-4a78-8e9c-e55dc89848e2-model-cache\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:46:31.571531 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:31.571481 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbj6h\" (UniqueName: \"kubernetes.io/projected/47a16951-1e0f-4a78-8e9c-e55dc89848e2-kube-api-access-mbj6h\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:46:32.291601 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:32.291526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-kjdth" event={"ID":"47a16951-1e0f-4a78-8e9c-e55dc89848e2","Type":"ContainerDied","Data":"aae6d2d256c418654ce4740f1e34b1dedc7c460b928bccfcf2f20954310d12ad"} Jul 01 18:46:32.291601 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:32.291549 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-kjdth" Jul 01 18:46:32.291601 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:32.291563 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aae6d2d256c418654ce4740f1e34b1dedc7c460b928bccfcf2f20954310d12ad" Jul 01 18:46:44.544696 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.544654 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k"] Jul 01 18:46:44.545152 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.544991 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47a16951-1e0f-4a78-8e9c-e55dc89848e2" containerName="s3-init" Jul 01 18:46:44.545152 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.545002 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="47a16951-1e0f-4a78-8e9c-e55dc89848e2" containerName="s3-init" Jul 01 18:46:44.545152 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.545020 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47a16951-1e0f-4a78-8e9c-e55dc89848e2" containerName="download-hf-model" Jul 01 18:46:44.545152 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.545026 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="47a16951-1e0f-4a78-8e9c-e55dc89848e2" containerName="download-hf-model" Jul 01 18:46:44.545152 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.545087 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="47a16951-1e0f-4a78-8e9c-e55dc89848e2" containerName="s3-init" Jul 01 18:46:44.548383 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.548334 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.553913 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.553858 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 01 18:46:44.554254 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.554237 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 18:46:44.555093 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.554931 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 01 18:46:44.555093 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.554966 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 01 18:46:44.555316 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.555298 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 01 18:46:44.555403 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.555337 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 01 18:46:44.555564 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.555546 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-qplkr\"" Jul 01 18:46:44.565231 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.565205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k"] Jul 01 18:46:44.683396 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.683349 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.683559 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.683403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.683559 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.683480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.683646 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.683560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.683646 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.683593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbkkt\" (UniqueName: \"kubernetes.io/projected/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-api-access-xbkkt\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.784263 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.784233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.784451 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.784274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.784451 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.784315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.784451 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.784359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.784451 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.784391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbkkt\" (UniqueName: \"kubernetes.io/projected/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-api-access-xbkkt\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.784664 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:46:44.784475 2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Jul 01 18:46:44.784664 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:46:44.784552 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls podName:4b8e9b12-0793-48ed-9d3d-1d855f2d81d6 nodeName:}" failed. No retries permitted until 2026-07-01 18:46:45.284528219 +0000 UTC m=+581.800119456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls") pod "isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" (UID: "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Jul 01 18:46:44.784784 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.784763 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.785021 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.785005 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.785067 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.785024 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.794864 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.794789 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbkkt\" (UniqueName: \"kubernetes.io/projected/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-api-access-xbkkt\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:44.952145 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.952106 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq"] Jul 01 18:46:44.955677 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.955660 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:44.958857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.958812 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 01 18:46:44.958857 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.958837 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 01 18:46:44.972197 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:44.972174 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq"] Jul 01 18:46:45.087510 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.087412 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.087510 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.087462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/95987d09-5a5b-46c7-a07a-f3312f69bade-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.087510 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.087488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/95987d09-5a5b-46c7-a07a-f3312f69bade-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.087755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.087576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtwkz\" (UniqueName: \"kubernetes.io/projected/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-api-access-dtwkz\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.087755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.087631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.188456 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.188423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.188640 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.188470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/95987d09-5a5b-46c7-a07a-f3312f69bade-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.188640 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.188498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/95987d09-5a5b-46c7-a07a-f3312f69bade-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.188640 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.188542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtwkz\" (UniqueName: \"kubernetes.io/projected/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-api-access-dtwkz\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.188640 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.188600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.189039 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.189006 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/95987d09-5a5b-46c7-a07a-f3312f69bade-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.189331 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.189305 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.189431 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.189305 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.191014 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.190992 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/95987d09-5a5b-46c7-a07a-f3312f69bade-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.198471 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.198447 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtwkz\" (UniqueName: \"kubernetes.io/projected/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-api-access-dtwkz\") pod \"isvc-xgboost-graph-predictor-555fc69d56-6hxpq\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.265476 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.265448 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:46:45.289444 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.289413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:45.289735 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.289713 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j"] Jul 01 18:46:45.291853 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.291814 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-875bcd459-8wb7k\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:45.293490 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.293475 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.298719 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.298687 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 01 18:46:45.298825 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.298727 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 01 18:46:45.318528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.318500 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j"] Jul 01 18:46:45.390568 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.390536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.390726 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.390587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.390726 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.390613 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-api-access-j5mtf\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.390726 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.390644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.390726 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.390672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9d478b59-375d-4cf4-a291-a1ecc867974e-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.405243 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.405207 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq"] Jul 01 18:46:45.408429 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:46:45.408404 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95987d09_5a5b_46c7_a07a_f3312f69bade.slice/crio-787a973fc5c771a4dffe0bec8df489238c43cf08f2495a6694b2785794757583 WatchSource:0}: Error finding container 787a973fc5c771a4dffe0bec8df489238c43cf08f2495a6694b2785794757583: Status 404 returned error can't find the container with id 787a973fc5c771a4dffe0bec8df489238c43cf08f2495a6694b2785794757583 Jul 01 18:46:45.458479 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.458456 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:46:45.491894 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.491828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.492063 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.491916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9d478b59-375d-4cf4-a291-a1ecc867974e-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.492063 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.491998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.492063 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.492058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.492221 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.492097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-api-access-j5mtf\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.492281 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:46:45.492216 2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Jul 01 18:46:45.492336 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:46:45.492284 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls podName:9d478b59-375d-4cf4-a291-a1ecc867974e nodeName:}" failed. No retries permitted until 2026-07-01 18:46:45.9922636 +0000 UTC m=+582.507854824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls") pod "isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" (UID: "9d478b59-375d-4cf4-a291-a1ecc867974e") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Jul 01 18:46:45.492406 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.492345 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9d478b59-375d-4cf4-a291-a1ecc867974e-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.492652 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.492627 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.492762 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.492741 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.501871 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.501828 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-api-access-j5mtf\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.618289 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.618254 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k"] Jul 01 18:46:45.622588 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:46:45.622560 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8e9b12_0793_48ed_9d3d_1d855f2d81d6.slice/crio-b241f586ad3379ef6f7a9b8b7d2e1096de0c0915b971bb3836ab09b29472cb9a WatchSource:0}: Error finding container b241f586ad3379ef6f7a9b8b7d2e1096de0c0915b971bb3836ab09b29472cb9a: Status 404 returned error can't find the container with id b241f586ad3379ef6f7a9b8b7d2e1096de0c0915b971bb3836ab09b29472cb9a Jul 01 18:46:45.996470 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.996392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:45.998721 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:45.998694 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:46.204205 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:46.204178 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:46:46.352038 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:46.352004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" event={"ID":"95987d09-5a5b-46c7-a07a-f3312f69bade","Type":"ContainerStarted","Data":"293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3"} Jul 01 18:46:46.352211 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:46.352072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" event={"ID":"95987d09-5a5b-46c7-a07a-f3312f69bade","Type":"ContainerStarted","Data":"787a973fc5c771a4dffe0bec8df489238c43cf08f2495a6694b2785794757583"} Jul 01 18:46:46.353828 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:46.353805 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" event={"ID":"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6","Type":"ContainerStarted","Data":"174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2"} Jul 01 18:46:46.353907 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:46.353869 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" event={"ID":"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6","Type":"ContainerStarted","Data":"b241f586ad3379ef6f7a9b8b7d2e1096de0c0915b971bb3836ab09b29472cb9a"} Jul 01 18:46:46.355536 ip-10-0-134-114 kubenswrapper[2562]: W0701 18:46:46.355515 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d478b59_375d_4cf4_a291_a1ecc867974e.slice/crio-109a61c895193ff350dd53d6692ab7a256582b32f3fd08c9af7cfd07db2647cd WatchSource:0}: Error finding container 109a61c895193ff350dd53d6692ab7a256582b32f3fd08c9af7cfd07db2647cd: Status 404 returned error can't find the container with id 109a61c895193ff350dd53d6692ab7a256582b32f3fd08c9af7cfd07db2647cd Jul 01 18:46:46.374012 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:46.373989 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j"] Jul 01 18:46:47.367949 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:47.367909 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" event={"ID":"9d478b59-375d-4cf4-a291-a1ecc867974e","Type":"ContainerStarted","Data":"f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea"} Jul 01 18:46:47.368352 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:47.368009 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" event={"ID":"9d478b59-375d-4cf4-a291-a1ecc867974e","Type":"ContainerStarted","Data":"109a61c895193ff350dd53d6692ab7a256582b32f3fd08c9af7cfd07db2647cd"} Jul 01 18:46:49.377449 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:49.377418 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerID="174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2" exitCode=0 Jul 01 18:46:49.377901 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:49.377490 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" event={"ID":"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6","Type":"ContainerDied","Data":"174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2"} Jul 01 18:46:49.379259 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:49.379226 2562 generic.go:356] "Generic (PLEG): container finished" podID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerID="293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3" exitCode=0 Jul 01 18:46:49.379403 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:49.379300 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" event={"ID":"95987d09-5a5b-46c7-a07a-f3312f69bade","Type":"ContainerDied","Data":"293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3"} Jul 01 18:46:50.394205 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:50.394167 2562 generic.go:356] "Generic (PLEG): container finished" podID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerID="f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea" exitCode=0 Jul 01 18:46:50.394779 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:46:50.394272 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" event={"ID":"9d478b59-375d-4cf4-a291-a1ecc867974e","Type":"ContainerDied","Data":"f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea"} Jul 01 18:47:17.418208 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:17.418181 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:47:17.418647 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:17.418628 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:47:17.422920 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:17.422898 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:47:17.423199 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:17.423177 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:47:17.566518 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:17.566487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" event={"ID":"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6","Type":"ContainerStarted","Data":"476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47"} Jul 01 18:47:18.575358 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:18.575270 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" event={"ID":"9d478b59-375d-4cf4-a291-a1ecc867974e","Type":"ContainerStarted","Data":"8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488"} Jul 01 18:47:18.577773 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:18.577741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" event={"ID":"95987d09-5a5b-46c7-a07a-f3312f69bade","Type":"ContainerStarted","Data":"ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec"} Jul 01 18:47:20.593747 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.593643 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" event={"ID":"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6","Type":"ContainerStarted","Data":"f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629"} Jul 01 18:47:20.594180 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.593866 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:47:20.595803 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.595780 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" event={"ID":"9d478b59-375d-4cf4-a291-a1ecc867974e","Type":"ContainerStarted","Data":"2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07"} Jul 01 18:47:20.595942 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.595829 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:47:20.595942 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.595876 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:47:20.597488 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.597464 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:47:20.598019 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.597999 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" event={"ID":"95987d09-5a5b-46c7-a07a-f3312f69bade","Type":"ContainerStarted","Data":"dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f"} Jul 01 18:47:20.598116 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.598055 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:47:20.598116 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.598089 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:47:20.599105 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.599076 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:47:20.625885 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.625821 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podStartSLOduration=2.503683693 podStartE2EDuration="36.625804494s" totalImagesPullingTime="30.688605651s" totalInitContainerRuntime="3.43351515s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:46:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:47:20.624151767 +0000 UTC m=+617.139743007" watchObservedRunningTime="2026-07-01 18:47:20.625804494 +0000 UTC m=+617.141395733" Jul 01 18:47:20.652375 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.652320 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podStartSLOduration=2.198072351 podStartE2EDuration="35.6523037s" totalImagesPullingTime="29.692649563s" totalInitContainerRuntime="3.761581786s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:46:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:47:20.650735356 +0000 UTC m=+617.166326595" watchObservedRunningTime="2026-07-01 18:47:20.6523037 +0000 UTC m=+617.167894941" Jul 01 18:47:20.681202 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:20.681145 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podStartSLOduration=2.248479405 podStartE2EDuration="36.681129868s" totalImagesPullingTime="30.702138797s" totalInitContainerRuntime="3.730511666s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:46:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:47:20.67847653 +0000 UTC m=+617.194067770" watchObservedRunningTime="2026-07-01 18:47:20.681129868 +0000 UTC m=+617.196721106" Jul 01 18:47:21.603043 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:21.602994 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:47:21.603528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:21.603096 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:47:21.603528 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:21.603409 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:47:21.604216 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:21.604192 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:47:22.607321 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:22.607285 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:47:26.607748 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:26.607640 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:47:26.608299 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:26.608271 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:47:26.608967 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:26.608947 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:47:26.609414 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:26.609391 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:47:27.612149 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:27.612116 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:47:27.612617 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:27.612591 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:47:36.608402 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:36.608361 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:47:36.609589 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:36.609564 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:47:37.613136 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:37.613089 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:47:46.609075 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:46.609035 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:47:46.609494 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:46.609466 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:47:47.613197 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:47.613154 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:47:56.608322 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:56.608283 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:47:56.609534 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:56.609508 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:47:57.612550 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:47:57.612510 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:48:06.608723 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:06.608679 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:48:06.609970 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:06.609938 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:48:07.613309 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:07.613267 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:48:16.608990 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:16.608946 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:48:16.609409 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:16.609382 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:48:17.613443 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:17.613400 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:48:26.608728 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:26.608701 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:48:26.609542 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:26.609521 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:48:27.613036 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:27.612995 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:48:28.956806 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:28.956763 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:48:36.610237 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:36.610201 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:48:38.958045 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:48:38.958013 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:49:14.634694 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.634611 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j"] Jul 01 18:49:14.635185 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.634954 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" containerID="cri-o://8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488" gracePeriod=30 Jul 01 18:49:14.635185 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.634988 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kube-rbac-proxy" containerID="cri-o://2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07" gracePeriod=30 Jul 01 18:49:14.713050 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.713021 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k"] Jul 01 18:49:14.713288 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.713263 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" containerID="cri-o://476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47" gracePeriod=30 Jul 01 18:49:14.713367 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.713300 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kube-rbac-proxy" containerID="cri-o://f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629" gracePeriod=30 Jul 01 18:49:14.869130 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.869098 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq"] Jul 01 18:49:14.869338 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.869316 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" containerID="cri-o://ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec" gracePeriod=30 Jul 01 18:49:14.869397 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:14.869367 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kube-rbac-proxy" containerID="cri-o://dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f" gracePeriod=30 Jul 01 18:49:15.082035 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:15.081999 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerID="f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629" exitCode=2 Jul 01 18:49:15.082208 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:15.082077 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" event={"ID":"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6","Type":"ContainerDied","Data":"f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629"} Jul 01 18:49:15.084024 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:15.084001 2562 generic.go:356] "Generic (PLEG): container finished" podID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerID="2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07" exitCode=2 Jul 01 18:49:15.084138 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:15.084073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" event={"ID":"9d478b59-375d-4cf4-a291-a1ecc867974e","Type":"ContainerDied","Data":"2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07"} Jul 01 18:49:15.085995 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:15.085977 2562 generic.go:356] "Generic (PLEG): container finished" podID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerID="dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f" exitCode=2 Jul 01 18:49:15.086121 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:15.086049 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" event={"ID":"95987d09-5a5b-46c7-a07a-f3312f69bade","Type":"ContainerDied","Data":"dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f"} Jul 01 18:49:16.603643 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:16.603594 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused" Jul 01 18:49:16.604053 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:16.603594 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.26:8643/healthz\": dial tcp 10.132.0.26:8643: connect: connection refused" Jul 01 18:49:16.608904 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:16.608876 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Jul 01 18:49:16.610129 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:16.610102 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 01 18:49:17.607626 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:17.607584 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.24:8643/healthz\": dial tcp 10.132.0.24:8643: connect: connection refused" Jul 01 18:49:18.716456 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.716429 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:49:18.832911 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.832808 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/95987d09-5a5b-46c7-a07a-f3312f69bade-proxy-tls\") pod \"95987d09-5a5b-46c7-a07a-f3312f69bade\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " Jul 01 18:49:18.832911 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.832865 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/95987d09-5a5b-46c7-a07a-f3312f69bade-kserve-provision-location\") pod \"95987d09-5a5b-46c7-a07a-f3312f69bade\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " Jul 01 18:49:18.832911 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.832895 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-rbac-proxy-sar-config\") pod \"95987d09-5a5b-46c7-a07a-f3312f69bade\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " Jul 01 18:49:18.833184 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.832961 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtwkz\" (UniqueName: \"kubernetes.io/projected/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-api-access-dtwkz\") pod \"95987d09-5a5b-46c7-a07a-f3312f69bade\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " Jul 01 18:49:18.833184 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.833021 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-cabundle-cert\") pod \"95987d09-5a5b-46c7-a07a-f3312f69bade\" (UID: \"95987d09-5a5b-46c7-a07a-f3312f69bade\") " Jul 01 18:49:18.833335 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.833238 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95987d09-5a5b-46c7-a07a-f3312f69bade-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "95987d09-5a5b-46c7-a07a-f3312f69bade" (UID: "95987d09-5a5b-46c7-a07a-f3312f69bade"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:49:18.833404 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.833325 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "95987d09-5a5b-46c7-a07a-f3312f69bade" (UID: "95987d09-5a5b-46c7-a07a-f3312f69bade"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:49:18.833404 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.833378 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "95987d09-5a5b-46c7-a07a-f3312f69bade" (UID: "95987d09-5a5b-46c7-a07a-f3312f69bade"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:49:18.835125 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.835104 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95987d09-5a5b-46c7-a07a-f3312f69bade-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "95987d09-5a5b-46c7-a07a-f3312f69bade" (UID: "95987d09-5a5b-46c7-a07a-f3312f69bade"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:49:18.835235 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.835216 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-api-access-dtwkz" (OuterVolumeSpecName: "kube-api-access-dtwkz") pod "95987d09-5a5b-46c7-a07a-f3312f69bade" (UID: "95987d09-5a5b-46c7-a07a-f3312f69bade"). InnerVolumeSpecName "kube-api-access-dtwkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:49:18.933914 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.933875 2562 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-cabundle-cert\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:18.933914 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.933904 2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/95987d09-5a5b-46c7-a07a-f3312f69bade-proxy-tls\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:18.933914 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.933914 2562 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/95987d09-5a5b-46c7-a07a-f3312f69bade-kserve-provision-location\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:18.933914 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.933922 2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:18.934180 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.933931 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtwkz\" (UniqueName: \"kubernetes.io/projected/95987d09-5a5b-46c7-a07a-f3312f69bade-kube-api-access-dtwkz\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:18.957509 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:18.957467 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Jul 01 18:49:19.104704 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.104668 2562 generic.go:356] "Generic (PLEG): container finished" podID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerID="ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec" exitCode=0 Jul 01 18:49:19.104900 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.104730 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" Jul 01 18:49:19.104900 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.104746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" event={"ID":"95987d09-5a5b-46c7-a07a-f3312f69bade","Type":"ContainerDied","Data":"ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec"} Jul 01 18:49:19.104900 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.104814 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq" event={"ID":"95987d09-5a5b-46c7-a07a-f3312f69bade","Type":"ContainerDied","Data":"787a973fc5c771a4dffe0bec8df489238c43cf08f2495a6694b2785794757583"} Jul 01 18:49:19.104900 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.104866 2562 scope.go:117] "RemoveContainer" containerID="dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f" Jul 01 18:49:19.113667 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.113650 2562 scope.go:117] "RemoveContainer" containerID="ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec" Jul 01 18:49:19.121254 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.121230 2562 scope.go:117] "RemoveContainer" containerID="293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3" Jul 01 18:49:19.129685 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.129661 2562 scope.go:117] "RemoveContainer" containerID="dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f" Jul 01 18:49:19.130043 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:19.130015 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f\": container with ID starting with dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f not found: ID does not exist" containerID="dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f" Jul 01 18:49:19.130139 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.130053 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f"} err="failed to get container status \"dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f\": rpc error: code = NotFound desc = could not find container \"dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f\": container with ID starting with dadac267a1a762b8635058394af5be19773caf5701b07222e4a6654bba64855f not found: ID does not exist" Jul 01 18:49:19.130139 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.130079 2562 scope.go:117] "RemoveContainer" containerID="ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec" Jul 01 18:49:19.130410 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:19.130380 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec\": container with ID starting with ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec not found: ID does not exist" containerID="ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec" Jul 01 18:49:19.130466 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.130420 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec"} err="failed to get container status \"ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec\": rpc error: code = NotFound desc = could not find container \"ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec\": container with ID starting with ba6fad563a4c656224497a2fc2a6fe17b29e986d0e1f902d923b2d5cbe4ca5ec not found: ID does not exist" Jul 01 18:49:19.130466 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.130442 2562 scope.go:117] "RemoveContainer" containerID="293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3" Jul 01 18:49:19.130561 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.130538 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq"] Jul 01 18:49:19.130714 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:19.130698 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3\": container with ID starting with 293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3 not found: ID does not exist" containerID="293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3" Jul 01 18:49:19.130755 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.130731 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3"} err="failed to get container status \"293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3\": rpc error: code = NotFound desc = could not find container \"293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3\": container with ID starting with 293356a86d6c1bc0712f2cfb67afd2976f17eb95b31beac3606fc4bd12e602e3 not found: ID does not exist" Jul 01 18:49:19.135092 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.135070 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-6hxpq"] Jul 01 18:49:19.961378 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:19.961344 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" path="/var/lib/kubelet/pods/95987d09-5a5b-46c7-a07a-f3312f69bade/volumes" Jul 01 18:49:20.536068 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.536043 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:49:20.650082 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650040 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9d478b59-375d-4cf4-a291-a1ecc867974e-kserve-provision-location\") pod \"9d478b59-375d-4cf4-a291-a1ecc867974e\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " Jul 01 18:49:20.650258 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650132 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-api-access-j5mtf\") pod \"9d478b59-375d-4cf4-a291-a1ecc867974e\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " Jul 01 18:49:20.650258 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls\") pod \"9d478b59-375d-4cf4-a291-a1ecc867974e\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " Jul 01 18:49:20.650258 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650195 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-cabundle-cert\") pod \"9d478b59-375d-4cf4-a291-a1ecc867974e\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " Jul 01 18:49:20.650258 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650226 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-rbac-proxy-sar-config\") pod \"9d478b59-375d-4cf4-a291-a1ecc867974e\" (UID: \"9d478b59-375d-4cf4-a291-a1ecc867974e\") " Jul 01 18:49:20.650480 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650412 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d478b59-375d-4cf4-a291-a1ecc867974e-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9d478b59-375d-4cf4-a291-a1ecc867974e" (UID: "9d478b59-375d-4cf4-a291-a1ecc867974e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:49:20.650536 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650501 2562 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9d478b59-375d-4cf4-a291-a1ecc867974e-kserve-provision-location\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.650588 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650559 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9d478b59-375d-4cf4-a291-a1ecc867974e" (UID: "9d478b59-375d-4cf4-a291-a1ecc867974e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:49:20.650767 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.650741 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9d478b59-375d-4cf4-a291-a1ecc867974e" (UID: "9d478b59-375d-4cf4-a291-a1ecc867974e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:49:20.652399 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.652369 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9d478b59-375d-4cf4-a291-a1ecc867974e" (UID: "9d478b59-375d-4cf4-a291-a1ecc867974e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:49:20.652552 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.652408 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-api-access-j5mtf" (OuterVolumeSpecName: "kube-api-access-j5mtf") pod "9d478b59-375d-4cf4-a291-a1ecc867974e" (UID: "9d478b59-375d-4cf4-a291-a1ecc867974e"). InnerVolumeSpecName "kube-api-access-j5mtf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:49:20.662088 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.662056 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:49:20.751708 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.751664 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls\") pod \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " Jul 01 18:49:20.751708 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.751712 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-cabundle-cert\") pod \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " Jul 01 18:49:20.751987 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.751761 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-rbac-proxy-sar-config\") pod \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " Jul 01 18:49:20.751987 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.751836 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kserve-provision-location\") pod \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " Jul 01 18:49:20.751987 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.751900 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbkkt\" (UniqueName: \"kubernetes.io/projected/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-api-access-xbkkt\") pod \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\" (UID: \"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6\") " Jul 01 18:49:20.752141 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.752131 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5mtf\" (UniqueName: \"kubernetes.io/projected/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-api-access-j5mtf\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.752203 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.752143 2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d478b59-375d-4cf4-a291-a1ecc867974e-proxy-tls\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.752203 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.752156 2562 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-cabundle-cert\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.752203 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.752178 2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9d478b59-375d-4cf4-a291-a1ecc867974e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.752203 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.752150 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" (UID: "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:49:20.752203 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.752181 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" (UID: "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:49:20.752357 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.752155 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" (UID: "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:49:20.753912 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.753876 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" (UID: "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:49:20.754008 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.753969 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-api-access-xbkkt" (OuterVolumeSpecName: "kube-api-access-xbkkt") pod "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" (UID: "4b8e9b12-0793-48ed-9d3d-1d855f2d81d6"). InnerVolumeSpecName "kube-api-access-xbkkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:49:20.852859 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.852812 2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-proxy-tls\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.852859 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.852860 2562 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-cabundle-cert\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.853047 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.852870 2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.853047 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.852881 2562 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kserve-provision-location\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:20.853047 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:20.852889 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbkkt\" (UniqueName: \"kubernetes.io/projected/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6-kube-api-access-xbkkt\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 18:49:21.116000 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.115912 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerID="476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47" exitCode=0 Jul 01 18:49:21.116000 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.115970 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" Jul 01 18:49:21.116000 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.115990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" event={"ID":"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6","Type":"ContainerDied","Data":"476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47"} Jul 01 18:49:21.116554 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.116061 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k" event={"ID":"4b8e9b12-0793-48ed-9d3d-1d855f2d81d6","Type":"ContainerDied","Data":"b241f586ad3379ef6f7a9b8b7d2e1096de0c0915b971bb3836ab09b29472cb9a"} Jul 01 18:49:21.116554 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.116099 2562 scope.go:117] "RemoveContainer" containerID="f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629" Jul 01 18:49:21.118129 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.118107 2562 generic.go:356] "Generic (PLEG): container finished" podID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerID="8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488" exitCode=0 Jul 01 18:49:21.118252 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.118171 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" Jul 01 18:49:21.118252 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.118185 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" event={"ID":"9d478b59-375d-4cf4-a291-a1ecc867974e","Type":"ContainerDied","Data":"8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488"} Jul 01 18:49:21.118252 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.118235 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j" event={"ID":"9d478b59-375d-4cf4-a291-a1ecc867974e","Type":"ContainerDied","Data":"109a61c895193ff350dd53d6692ab7a256582b32f3fd08c9af7cfd07db2647cd"} Jul 01 18:49:21.124496 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.124476 2562 scope.go:117] "RemoveContainer" containerID="476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47" Jul 01 18:49:21.132541 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.132521 2562 scope.go:117] "RemoveContainer" containerID="174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2" Jul 01 18:49:21.140208 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.140188 2562 scope.go:117] "RemoveContainer" containerID="f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629" Jul 01 18:49:21.140488 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:21.140462 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629\": container with ID starting with f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629 not found: ID does not exist" containerID="f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629" Jul 01 18:49:21.140560 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.140491 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629"} err="failed to get container status \"f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629\": rpc error: code = NotFound desc = could not find container \"f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629\": container with ID starting with f091c48684b5e22213baf5054902319902a25922c46be469cc61fa8b77bb3629 not found: ID does not exist" Jul 01 18:49:21.140560 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.140508 2562 scope.go:117] "RemoveContainer" containerID="476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47" Jul 01 18:49:21.140649 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.140563 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k"] Jul 01 18:49:21.140777 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:21.140758 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47\": container with ID starting with 476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47 not found: ID does not exist" containerID="476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47" Jul 01 18:49:21.140817 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.140784 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47"} err="failed to get container status \"476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47\": rpc error: code = NotFound desc = could not find container \"476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47\": container with ID starting with 476e1d149f8a4657bbb2f0a2aab703dff10e1719f579ecdf9afad65ae3b43c47 not found: ID does not exist" Jul 01 18:49:21.140817 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.140800 2562 scope.go:117] "RemoveContainer" containerID="174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2" Jul 01 18:49:21.141043 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:21.141024 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2\": container with ID starting with 174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2 not found: ID does not exist" containerID="174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2" Jul 01 18:49:21.141089 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.141047 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2"} err="failed to get container status \"174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2\": rpc error: code = NotFound desc = could not find container \"174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2\": container with ID starting with 174d490c919340f5a0dae400b6b4c614ee5bd3fa8801865539a034c4ee5955b2 not found: ID does not exist" Jul 01 18:49:21.141089 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.141058 2562 scope.go:117] "RemoveContainer" containerID="2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07" Jul 01 18:49:21.144585 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.144566 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-875bcd459-8wb7k"] Jul 01 18:49:21.148985 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.148969 2562 scope.go:117] "RemoveContainer" containerID="8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488" Jul 01 18:49:21.156677 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.156657 2562 scope.go:117] "RemoveContainer" containerID="f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea" Jul 01 18:49:21.157071 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.157055 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j"] Jul 01 18:49:21.161135 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.161115 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4v96j"] Jul 01 18:49:21.164076 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.164060 2562 scope.go:117] "RemoveContainer" containerID="2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07" Jul 01 18:49:21.164358 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:21.164336 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07\": container with ID starting with 2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07 not found: ID does not exist" containerID="2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07" Jul 01 18:49:21.164422 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.164362 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07"} err="failed to get container status \"2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07\": rpc error: code = NotFound desc = could not find container \"2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07\": container with ID starting with 2565a47ec0456c97c5a78c2602880e5dceb4ceeb6a246760217a0bbb1ce80a07 not found: ID does not exist" Jul 01 18:49:21.164422 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.164381 2562 scope.go:117] "RemoveContainer" containerID="8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488" Jul 01 18:49:21.164612 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:21.164596 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488\": container with ID starting with 8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488 not found: ID does not exist" containerID="8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488" Jul 01 18:49:21.164655 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.164618 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488"} err="failed to get container status \"8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488\": rpc error: code = NotFound desc = could not find container \"8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488\": container with ID starting with 8a5b04c822c88bb6b21b3098ddeaa7742f73900c6a2520a684a6879272a4f488 not found: ID does not exist" Jul 01 18:49:21.164655 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.164634 2562 scope.go:117] "RemoveContainer" containerID="f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea" Jul 01 18:49:21.164875 ip-10-0-134-114 kubenswrapper[2562]: E0701 18:49:21.164859 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea\": container with ID starting with f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea not found: ID does not exist" containerID="f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea" Jul 01 18:49:21.164933 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.164878 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea"} err="failed to get container status \"f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea\": rpc error: code = NotFound desc = could not find container \"f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea\": container with ID starting with f36efb3016fdd623045536801b4a0675906222187427a14b2f7d14fed2f4e0ea not found: ID does not exist" Jul 01 18:49:21.960854 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.960817 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" path="/var/lib/kubelet/pods/4b8e9b12-0793-48ed-9d3d-1d855f2d81d6/volumes" Jul 01 18:49:21.961339 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:49:21.961325 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" path="/var/lib/kubelet/pods/9d478b59-375d-4cf4-a291-a1ecc867974e/volumes" Jul 01 18:52:17.444638 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:52:17.444554 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:52:17.446513 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:52:17.446490 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:52:17.448705 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:52:17.448685 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:52:17.450430 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:52:17.450413 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:57:17.472529 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:57:17.472501 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:57:17.475591 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:57:17.475571 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 18:57:17.476661 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:57:17.476639 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 18:57:17.479446 ip-10-0-134-114 kubenswrapper[2562]: I0701 18:57:17.479425 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:02:17.498476 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:02:17.498359 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:02:17.502638 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:02:17.501681 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:02:17.503015 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:02:17.502990 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:02:17.506315 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:02:17.506294 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:07:17.524110 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:07:17.524005 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:07:17.528251 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:07:17.528040 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:07:17.528251 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:07:17.528053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:07:17.531907 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:07:17.531889 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:12:17.549208 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:12:17.549089 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:12:17.553245 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:12:17.553226 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:12:17.554958 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:12:17.554937 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:12:17.558856 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:12:17.558818 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:17:17.573657 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:17:17.573544 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:17:17.578181 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:17:17.578161 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:17:17.584624 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:17:17.584606 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:17:17.588411 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:17:17.588394 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:22:17.599147 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:22:17.599029 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:22:17.603300 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:22:17.603279 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:22:17.610647 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:22:17.610629 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:22:17.614654 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:22:17.614637 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:26:38.810825 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.810744 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bs9j6/must-gather-fwsf9"] Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811064 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="storage-initializer" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811076 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="storage-initializer" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811092 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811097 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811105 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811111 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811119 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811124 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811134 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="storage-initializer" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811139 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="storage-initializer" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811145 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811150 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811157 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811162 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811169 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="storage-initializer" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811175 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="storage-initializer" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811183 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811188 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811240 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811250 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811256 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b8e9b12-0793-48ed-9d3d-1d855f2d81d6" containerName="kserve-container" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811262 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kube-rbac-proxy" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811267 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="95987d09-5a5b-46c7-a07a-f3312f69bade" containerName="kserve-container" Jul 01 19:26:38.811287 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.811272 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d478b59-375d-4cf4-a291-a1ecc867974e" containerName="kserve-container" Jul 01 19:26:38.814401 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.814385 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:26:38.816893 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.816872 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bs9j6\"/\"openshift-service-ca.crt\"" Jul 01 19:26:38.817068 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.816890 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bs9j6\"/\"kube-root-ca.crt\"" Jul 01 19:26:38.817439 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.817421 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bs9j6\"/\"default-dockercfg-wf4cg\"" Jul 01 19:26:38.822312 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.822288 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bs9j6/must-gather-fwsf9"] Jul 01 19:26:38.938046 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.938008 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-must-gather-output\") pod \"must-gather-fwsf9\" (UID: \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\") " pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:26:38.938046 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:38.938054 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47ll5\" (UniqueName: \"kubernetes.io/projected/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-kube-api-access-47ll5\") pod \"must-gather-fwsf9\" (UID: \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\") " pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:26:39.039545 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:39.039510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-must-gather-output\") pod \"must-gather-fwsf9\" (UID: \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\") " pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:26:39.039545 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:39.039550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47ll5\" (UniqueName: \"kubernetes.io/projected/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-kube-api-access-47ll5\") pod \"must-gather-fwsf9\" (UID: \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\") " pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:26:39.039907 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:39.039876 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-must-gather-output\") pod \"must-gather-fwsf9\" (UID: \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\") " pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:26:39.048619 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:39.048598 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47ll5\" (UniqueName: \"kubernetes.io/projected/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-kube-api-access-47ll5\") pod \"must-gather-fwsf9\" (UID: \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\") " pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:26:39.124017 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:39.123974 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:26:39.251337 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:39.251303 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bs9j6/must-gather-fwsf9"] Jul 01 19:26:39.253886 ip-10-0-134-114 kubenswrapper[2562]: W0701 19:26:39.253830 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40afbfa9_fa9a_4aef_8662_2cfcc96f0ce2.slice/crio-1f7e0540a13836b58ef7039d91a84b6614967bdc23ea0e3221fe631b7a513ac2 WatchSource:0}: Error finding container 1f7e0540a13836b58ef7039d91a84b6614967bdc23ea0e3221fe631b7a513ac2: Status 404 returned error can't find the container with id 1f7e0540a13836b58ef7039d91a84b6614967bdc23ea0e3221fe631b7a513ac2 Jul 01 19:26:39.255598 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:39.255581 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 19:26:39.758785 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:39.758746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" event={"ID":"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2","Type":"ContainerStarted","Data":"1f7e0540a13836b58ef7039d91a84b6614967bdc23ea0e3221fe631b7a513ac2"} Jul 01 19:26:46.795338 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:46.795293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" event={"ID":"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2","Type":"ContainerStarted","Data":"4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590"} Jul 01 19:26:46.795776 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:46.795350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" event={"ID":"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2","Type":"ContainerStarted","Data":"8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5"} Jul 01 19:26:46.811667 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:26:46.811609 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" podStartSLOduration=2.162794036 podStartE2EDuration="8.811594051s" totalImagesPullingTime="6.648800015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:26:46.811006695 +0000 UTC m=+2983.326597939" watchObservedRunningTime="2026-07-01 19:26:46.811594051 +0000 UTC m=+2983.327185302" Jul 01 19:27:08.895274 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:08.895237 2562 generic.go:356] "Generic (PLEG): container finished" podID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerID="8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5" exitCode=0 Jul 01 19:27:08.895867 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:08.895311 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" event={"ID":"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2","Type":"ContainerDied","Data":"8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5"} Jul 01 19:27:08.895867 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:08.895707 2562 scope.go:117] "RemoveContainer" containerID="8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5" Jul 01 19:27:09.537607 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:09.537579 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bs9j6_must-gather-fwsf9_40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2/gather/0.log" Jul 01 19:27:10.153099 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.153063 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gprgv/must-gather-s6sln"] Jul 01 19:27:10.155711 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.155689 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gprgv/must-gather-s6sln" Jul 01 19:27:10.158137 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.158110 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gprgv\"/\"kube-root-ca.crt\"" Jul 01 19:27:10.158257 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.158167 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gprgv\"/\"default-dockercfg-9l27g\"" Jul 01 19:27:10.158257 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.158206 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gprgv\"/\"openshift-service-ca.crt\"" Jul 01 19:27:10.166770 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.166740 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gprgv/must-gather-s6sln"] Jul 01 19:27:10.229017 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.228976 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqfq4\" (UniqueName: \"kubernetes.io/projected/a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1-kube-api-access-mqfq4\") pod \"must-gather-s6sln\" (UID: \"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1\") " pod="openshift-must-gather-gprgv/must-gather-s6sln" Jul 01 19:27:10.229198 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.229024 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1-must-gather-output\") pod \"must-gather-s6sln\" (UID: \"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1\") " pod="openshift-must-gather-gprgv/must-gather-s6sln" Jul 01 19:27:10.330128 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.330080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqfq4\" (UniqueName: \"kubernetes.io/projected/a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1-kube-api-access-mqfq4\") pod \"must-gather-s6sln\" (UID: \"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1\") " pod="openshift-must-gather-gprgv/must-gather-s6sln" Jul 01 19:27:10.330128 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.330133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1-must-gather-output\") pod \"must-gather-s6sln\" (UID: \"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1\") " pod="openshift-must-gather-gprgv/must-gather-s6sln" Jul 01 19:27:10.330555 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.330537 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1-must-gather-output\") pod \"must-gather-s6sln\" (UID: \"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1\") " pod="openshift-must-gather-gprgv/must-gather-s6sln" Jul 01 19:27:10.338688 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.338661 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqfq4\" (UniqueName: \"kubernetes.io/projected/a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1-kube-api-access-mqfq4\") pod \"must-gather-s6sln\" (UID: \"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1\") " pod="openshift-must-gather-gprgv/must-gather-s6sln" Jul 01 19:27:10.469801 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.469709 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gprgv/must-gather-s6sln" Jul 01 19:27:10.597588 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.597556 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gprgv/must-gather-s6sln"] Jul 01 19:27:10.600127 ip-10-0-134-114 kubenswrapper[2562]: W0701 19:27:10.600099 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0e50ffc_4ab7_4f0f_b0cd_c381909d7fc1.slice/crio-24a5111a70620a903f6a3ee3518d4c0e9dd8975475863e2bb692bfc677fd4071 WatchSource:0}: Error finding container 24a5111a70620a903f6a3ee3518d4c0e9dd8975475863e2bb692bfc677fd4071: Status 404 returned error can't find the container with id 24a5111a70620a903f6a3ee3518d4c0e9dd8975475863e2bb692bfc677fd4071 Jul 01 19:27:10.903325 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:10.903287 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gprgv/must-gather-s6sln" event={"ID":"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1","Type":"ContainerStarted","Data":"24a5111a70620a903f6a3ee3518d4c0e9dd8975475863e2bb692bfc677fd4071"} Jul 01 19:27:11.912378 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:11.912343 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gprgv/must-gather-s6sln" event={"ID":"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1","Type":"ContainerStarted","Data":"37b84a01383dab6ebf2a70e4affdede26d20bb51d8bc1fe6183ac4897ab33014"} Jul 01 19:27:11.912802 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:11.912391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gprgv/must-gather-s6sln" event={"ID":"a0e50ffc-4ab7-4f0f-b0cd-c381909d7fc1","Type":"ContainerStarted","Data":"c957a1cbaacd90690af8bb1e741123740cd4dfad08432339febd408fa66ee57f"} Jul 01 19:27:11.931056 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:11.930995 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gprgv/must-gather-s6sln" podStartSLOduration=1.291324087 podStartE2EDuration="1.930974786s" totalImagesPullingTime="639.650699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:27:11.929803394 +0000 UTC m=+3008.445394634" watchObservedRunningTime="2026-07-01 19:27:11.930974786 +0000 UTC m=+3008.446566028" Jul 01 19:27:12.629262 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:12.629227 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j8lms_a5932d17-9b81-4e7e-94ce-1db0cded279d/global-pull-secret-syncer/0.log" Jul 01 19:27:12.826676 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:12.826620 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l8k74_d36698d3-80dd-4298-92ff-30f664e015fc/konnectivity-agent/0.log" Jul 01 19:27:12.879475 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:12.879387 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-114.ec2.internal_2d9920d8a67dc75b8a30a40184be48bc/haproxy/0.log" Jul 01 19:27:14.984516 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:14.984468 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-bs9j6/must-gather-fwsf9"] Jul 01 19:27:14.985163 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:14.984720 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerName="copy" containerID="cri-o://4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590" gracePeriod=2 Jul 01 19:27:14.987275 ip-10-0-134-114 kubenswrapper[2562]: E0701 19:27:14.987238 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-fwsf9\" is forbidden: User \"system:node:ip-10-0-134-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bs9j6\": no relationship found between node 'ip-10-0-134-114.ec2.internal' and this object" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:27:14.989919 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:14.989894 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-bs9j6/must-gather-fwsf9"] Jul 01 19:27:15.421392 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.420023 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bs9j6_must-gather-fwsf9_40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2/copy/0.log" Jul 01 19:27:15.424266 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.424238 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:27:15.431237 ip-10-0-134-114 kubenswrapper[2562]: E0701 19:27:15.430079 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-fwsf9\" is forbidden: User \"system:node:ip-10-0-134-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bs9j6\": no relationship found between node 'ip-10-0-134-114.ec2.internal' and this object" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:27:15.589290 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.587021 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47ll5\" (UniqueName: \"kubernetes.io/projected/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-kube-api-access-47ll5\") pod \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\" (UID: \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\") " Jul 01 19:27:15.589290 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.587107 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-must-gather-output\") pod \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\" (UID: \"40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2\") " Jul 01 19:27:15.589290 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.588910 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" (UID: "40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:27:15.591603 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.591559 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-kube-api-access-47ll5" (OuterVolumeSpecName: "kube-api-access-47ll5") pod "40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" (UID: "40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2"). InnerVolumeSpecName "kube-api-access-47ll5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:27:15.688458 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.688385 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47ll5\" (UniqueName: \"kubernetes.io/projected/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-kube-api-access-47ll5\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 19:27:15.688458 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.688425 2562 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2-must-gather-output\") on node \"ip-10-0-134-114.ec2.internal\" DevicePath \"\"" Jul 01 19:27:15.934069 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.934034 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bs9j6_must-gather-fwsf9_40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2/copy/0.log" Jul 01 19:27:15.934867 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.934822 2562 generic.go:356] "Generic (PLEG): container finished" podID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerID="4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590" exitCode=143 Jul 01 19:27:15.935056 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.934927 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:27:15.935205 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.934960 2562 scope.go:117] "RemoveContainer" containerID="4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590" Jul 01 19:27:15.937627 ip-10-0-134-114 kubenswrapper[2562]: E0701 19:27:15.937575 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-fwsf9\" is forbidden: User \"system:node:ip-10-0-134-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bs9j6\": no relationship found between node 'ip-10-0-134-114.ec2.internal' and this object" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:27:15.947581 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.947206 2562 scope.go:117] "RemoveContainer" containerID="8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5" Jul 01 19:27:15.952813 ip-10-0-134-114 kubenswrapper[2562]: E0701 19:27:15.952716 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-fwsf9\" is forbidden: User \"system:node:ip-10-0-134-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bs9j6\": no relationship found between node 'ip-10-0-134-114.ec2.internal' and this object" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" pod="openshift-must-gather-bs9j6/must-gather-fwsf9" Jul 01 19:27:15.966738 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.966691 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" path="/var/lib/kubelet/pods/40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2/volumes" Jul 01 19:27:15.976590 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.976450 2562 scope.go:117] "RemoveContainer" containerID="4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590" Jul 01 19:27:15.977243 ip-10-0-134-114 kubenswrapper[2562]: E0701 19:27:15.977083 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590\": container with ID starting with 4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590 not found: ID does not exist" containerID="4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590" Jul 01 19:27:15.977243 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.977124 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590"} err="failed to get container status \"4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590\": rpc error: code = NotFound desc = could not find container \"4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590\": container with ID starting with 4c84c817410f590ad8d65061592e09b4bbc00d4a31564a95c5d3f4b2593bc590 not found: ID does not exist" Jul 01 19:27:15.977243 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.977148 2562 scope.go:117] "RemoveContainer" containerID="8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5" Jul 01 19:27:15.977505 ip-10-0-134-114 kubenswrapper[2562]: E0701 19:27:15.977478 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5\": container with ID starting with 8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5 not found: ID does not exist" containerID="8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5" Jul 01 19:27:15.977575 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:15.977517 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5"} err="failed to get container status \"8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5\": rpc error: code = NotFound desc = could not find container \"8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5\": container with ID starting with 8156e70a55c2131dee13ccdf084f68dbd398683042802e9b3d005da08f668fa5 not found: ID does not exist" Jul 01 19:27:17.236908 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.236876 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_25b068d9-24e4-4f8f-94d6-609b5e26004f/alertmanager/0.log" Jul 01 19:27:17.268313 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.268273 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_25b068d9-24e4-4f8f-94d6-609b5e26004f/config-reloader/0.log" Jul 01 19:27:17.298176 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.296321 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_25b068d9-24e4-4f8f-94d6-609b5e26004f/kube-rbac-proxy-web/0.log" Jul 01 19:27:17.324068 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.324034 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_25b068d9-24e4-4f8f-94d6-609b5e26004f/kube-rbac-proxy/0.log" Jul 01 19:27:17.350320 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.350284 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_25b068d9-24e4-4f8f-94d6-609b5e26004f/kube-rbac-proxy-metric/0.log" Jul 01 19:27:17.377397 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.377366 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_25b068d9-24e4-4f8f-94d6-609b5e26004f/prom-label-proxy/0.log" Jul 01 19:27:17.410670 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.410641 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_25b068d9-24e4-4f8f-94d6-609b5e26004f/init-config-reloader/0.log" Jul 01 19:27:17.504336 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.504194 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-mkf4s_2ba158e2-d820-45d2-b2ad-d49801b0a2af/kube-state-metrics/0.log" Jul 01 19:27:17.539586 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.539537 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-mkf4s_2ba158e2-d820-45d2-b2ad-d49801b0a2af/kube-rbac-proxy-main/0.log" Jul 01 19:27:17.563863 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.563818 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-mkf4s_2ba158e2-d820-45d2-b2ad-d49801b0a2af/kube-rbac-proxy-self/0.log" Jul 01 19:27:17.642472 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.642351 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:27:17.649990 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.649965 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:27:17.655428 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.655395 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:27:17.665233 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.665199 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:27:17.872235 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.872203 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wmlhj_a95d3879-6a5f-48a9-b564-704fb4068c32/node-exporter/0.log" Jul 01 19:27:17.898072 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.898040 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wmlhj_a95d3879-6a5f-48a9-b564-704fb4068c32/kube-rbac-proxy/0.log" Jul 01 19:27:17.921714 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.921678 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wmlhj_a95d3879-6a5f-48a9-b564-704fb4068c32/init-textfile/0.log" Jul 01 19:27:17.956897 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.956830 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-4xpv6_cdc8ae1e-8518-44d7-befb-97872eb358d9/kube-rbac-proxy-main/0.log" Jul 01 19:27:17.984458 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:17.984424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-4xpv6_cdc8ae1e-8518-44d7-befb-97872eb358d9/kube-rbac-proxy-self/0.log" Jul 01 19:27:18.011458 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:18.011424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-4xpv6_cdc8ae1e-8518-44d7-befb-97872eb358d9/openshift-state-metrics/0.log" Jul 01 19:27:18.069431 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:18.069344 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff98045f-c140-4839-be98-991560af691a/prometheus/0.log" Jul 01 19:27:18.091934 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:18.091901 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff98045f-c140-4839-be98-991560af691a/config-reloader/0.log" Jul 01 19:27:18.116253 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:18.116212 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff98045f-c140-4839-be98-991560af691a/thanos-sidecar/0.log" Jul 01 19:27:18.140116 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:18.140062 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff98045f-c140-4839-be98-991560af691a/kube-rbac-proxy-web/0.log" Jul 01 19:27:18.165453 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:18.165423 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff98045f-c140-4839-be98-991560af691a/kube-rbac-proxy/0.log" Jul 01 19:27:18.189782 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:18.189748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff98045f-c140-4839-be98-991560af691a/kube-rbac-proxy-thanos/0.log" Jul 01 19:27:18.214491 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:18.214456 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ff98045f-c140-4839-be98-991560af691a/init-config-reloader/0.log" Jul 01 19:27:19.879921 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:19.879885 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cdb5968b9-4v4xx_b960b80f-9641-425c-8e38-ca1d294ceb98/networking-console-plugin/0.log" Jul 01 19:27:20.423011 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:20.422949 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/1.log" Jul 01 19:27:20.428938 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:20.428854 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-lfn8v_7f8b9fef-153e-4b4c-b13d-43ac17462991/console-operator/2.log" Jul 01 19:27:20.875526 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:20.875469 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-2nvtf_23ddf9c4-6b4e-4dc1-94fd-6e556b68de76/download-server/0.log" Jul 01 19:27:21.150937 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.150833 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp"] Jul 01 19:27:21.151928 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.151890 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerName="copy" Jul 01 19:27:21.152118 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.152094 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerName="copy" Jul 01 19:27:21.152244 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.152139 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerName="gather" Jul 01 19:27:21.152244 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.152149 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerName="gather" Jul 01 19:27:21.152244 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.152242 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerName="gather" Jul 01 19:27:21.152399 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.152258 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="40afbfa9-fa9a-4aef-8662-2cfcc96f0ce2" containerName="copy" Jul 01 19:27:21.156758 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.156735 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.167716 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.167692 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp"] Jul 01 19:27:21.342801 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.342751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-podres\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.343020 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.342813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-sys\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.343020 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.342870 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-proc\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.343020 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.342953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6zlk\" (UniqueName: \"kubernetes.io/projected/16316f87-64a3-4331-88e6-a40fd8d0e57e-kube-api-access-t6zlk\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.343020 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.343001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-lib-modules\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.431651 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.431576 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-g4gg8_50112456-daf6-4609-99d3-87004be60195/volume-data-source-validator/0.log" Jul 01 19:27:21.443434 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-podres\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.443624 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443443 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-sys\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.443624 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-proc\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.443624 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6zlk\" (UniqueName: \"kubernetes.io/projected/16316f87-64a3-4331-88e6-a40fd8d0e57e-kube-api-access-t6zlk\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.443624 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443541 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-sys\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.443624 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443570 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-podres\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.443624 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-lib-modules\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.443892 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443636 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-proc\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.443892 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.443680 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16316f87-64a3-4331-88e6-a40fd8d0e57e-lib-modules\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.452624 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.452597 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6zlk\" (UniqueName: \"kubernetes.io/projected/16316f87-64a3-4331-88e6-a40fd8d0e57e-kube-api-access-t6zlk\") pod \"perf-node-gather-daemonset-kxxdp\" (UID: \"16316f87-64a3-4331-88e6-a40fd8d0e57e\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.470874 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.470812 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.632192 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.632163 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp"] Jul 01 19:27:21.639788 ip-10-0-134-114 kubenswrapper[2562]: W0701 19:27:21.639370 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod16316f87_64a3_4331_88e6_a40fd8d0e57e.slice/crio-4f35d71296608dc4fe96f8c18659023559113dd9511567f85068ad7830823351 WatchSource:0}: Error finding container 4f35d71296608dc4fe96f8c18659023559113dd9511567f85068ad7830823351: Status 404 returned error can't find the container with id 4f35d71296608dc4fe96f8c18659023559113dd9511567f85068ad7830823351 Jul 01 19:27:21.978310 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.978223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" event={"ID":"16316f87-64a3-4331-88e6-a40fd8d0e57e","Type":"ContainerStarted","Data":"a54b7509285762b2ec785955789461a28c84bddb63ef4f453f533d8dcd40c79b"} Jul 01 19:27:21.978310 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.978288 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" event={"ID":"16316f87-64a3-4331-88e6-a40fd8d0e57e","Type":"ContainerStarted","Data":"4f35d71296608dc4fe96f8c18659023559113dd9511567f85068ad7830823351"} Jul 01 19:27:21.978553 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.978403 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:21.997182 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:21.997120 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" podStartSLOduration=0.997099319 podStartE2EDuration="997.099319ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:27:21.994924589 +0000 UTC m=+3018.510515828" watchObservedRunningTime="2026-07-01 19:27:21.997099319 +0000 UTC m=+3018.512690560" Jul 01 19:27:22.385075 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:22.385046 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wz7b8_b3eea556-5056-460c-809f-31ef7e9f333c/dns/0.log" Jul 01 19:27:22.410035 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:22.410006 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wz7b8_b3eea556-5056-460c-809f-31ef7e9f333c/kube-rbac-proxy/0.log" Jul 01 19:27:22.479708 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:22.479676 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fcb9x_2f250d29-e5fb-4c99-8662-29dddf0e37c5/dns-node-resolver/0.log" Jul 01 19:27:23.064567 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:23.064535 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sh9jq_e4c2815f-c0b3-4524-b6ce-96d3c5aba855/node-ca/0.log" Jul 01 19:27:24.227817 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:24.227789 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-59bd7_87032a95-9729-4094-a901-5ac3827cc3e1/serve-healthcheck-canary/0.log" Jul 01 19:27:24.761347 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:24.761316 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rzrfw_99cabad8-b09b-48f1-a331-7325be9882c0/kube-rbac-proxy/0.log" Jul 01 19:27:24.782887 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:24.782859 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rzrfw_99cabad8-b09b-48f1-a331-7325be9882c0/exporter/0.log" Jul 01 19:27:24.805208 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:24.805176 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rzrfw_99cabad8-b09b-48f1-a331-7325be9882c0/extractor/0.log" Jul 01 19:27:27.221860 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:27.221819 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-kjdth_47a16951-1e0f-4a78-8e9c-e55dc89848e2/s3-init/0.log" Jul 01 19:27:27.243409 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:27.243384 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-kjdth_47a16951-1e0f-4a78-8e9c-e55dc89848e2/download-hf-model/0.log" Jul 01 19:27:27.993175 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:27.993143 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-kxxdp" Jul 01 19:27:31.832880 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:31.832832 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-kvzr6_9e391929-8cd1-4e44-a07f-724feb9977b5/migrator/0.log" Jul 01 19:27:31.858399 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:31.858370 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-kvzr6_9e391929-8cd1-4e44-a07f-724feb9977b5/graceful-termination/0.log" Jul 01 19:27:33.154967 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.154924 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6srdv_dd36d396-98d4-4666-9a09-c575ead013da/kube-multus/0.log" Jul 01 19:27:33.600297 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.600255 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t9995_315b7908-1f18-4b3f-800d-05cf10e7da1e/kube-multus-additional-cni-plugins/0.log" Jul 01 19:27:33.633122 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.633088 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t9995_315b7908-1f18-4b3f-800d-05cf10e7da1e/egress-router-binary-copy/0.log" Jul 01 19:27:33.656082 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.656052 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t9995_315b7908-1f18-4b3f-800d-05cf10e7da1e/cni-plugins/0.log" Jul 01 19:27:33.680276 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.680253 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t9995_315b7908-1f18-4b3f-800d-05cf10e7da1e/bond-cni-plugin/0.log" Jul 01 19:27:33.704219 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.704191 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t9995_315b7908-1f18-4b3f-800d-05cf10e7da1e/routeoverride-cni/0.log" Jul 01 19:27:33.729351 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.729325 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t9995_315b7908-1f18-4b3f-800d-05cf10e7da1e/whereabouts-cni-bincopy/0.log" Jul 01 19:27:33.754592 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.754567 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t9995_315b7908-1f18-4b3f-800d-05cf10e7da1e/whereabouts-cni/0.log" Jul 01 19:27:33.859728 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.859655 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-npbg7_d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973/network-metrics-daemon/0.log" Jul 01 19:27:33.881711 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:33.881684 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-npbg7_d53c4c6a-ac5c-43f9-8ca4-181fdc7d4973/kube-rbac-proxy/0.log" Jul 01 19:27:34.909473 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:34.909447 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-controller/0.log" Jul 01 19:27:34.932122 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:34.932096 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/1.log" Jul 01 19:27:34.932704 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:34.932679 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovn-acl-logging/0.log" Jul 01 19:27:34.951365 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:34.951346 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/kube-rbac-proxy-node/0.log" Jul 01 19:27:34.977546 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:34.977522 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/kube-rbac-proxy-ovn-metrics/0.log" Jul 01 19:27:35.000438 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:35.000413 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/northd/0.log" Jul 01 19:27:35.022583 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:35.022553 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/nbdb/0.log" Jul 01 19:27:35.045032 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:35.044998 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/sbdb/0.log" Jul 01 19:27:35.091667 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:35.091636 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d9nnh_264efc66-6510-4a19-94f2-908be89caa25/ovnkube-controller/0.log" Jul 01 19:27:36.358636 ip-10-0-134-114 kubenswrapper[2562]: I0701 19:27:36.358603 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-67lhc_b40e8345-4927-47e8-8cd9-dba01f13a362/network-check-target-container/0.log"