Jul 08 16:32:23.232282 ip-10-0-130-254 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 16:32:23.232297 ip-10-0-130-254 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 16:32:23.232307 ip-10-0-130-254 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 16:32:23.232679 ip-10-0-130-254 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 16:32:33.255544 ip-10-0-130-254 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 16:32:33.255568 ip-10-0-130-254 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cbfcbf28ca82470aab5791629733a052 -- Jul 08 16:34:33.111316 ip-10-0-130-254 systemd[1]: Starting Kubernetes Kubelet... Jul 08 16:34:33.464503 ip-10-0-130-254 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 16:34:33.464503 ip-10-0-130-254 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 16:34:33.464503 ip-10-0-130-254 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 16:34:33.464503 ip-10-0-130-254 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 16:34:33.464503 ip-10-0-130-254 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 16:34:33.466693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.466597 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 16:34:33.471037 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471017 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:34:33.471037 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471036 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471040 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471043 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471047 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471049 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471052 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471054 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471057 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471060 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471062 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471065 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471068 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471071 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471074 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471077 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471079 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471082 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471084 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471087 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471089 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:34:33.471112 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471093 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471096 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471099 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471102 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471104 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471107 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471109 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471112 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471115 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471118 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471121 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471125 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471129 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471132 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471134 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471137 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471139 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471144 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471148 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:34:33.471592 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471151 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471156 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471159 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471162 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471165 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471167 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471170 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471173 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471175 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471177 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471180 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471182 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471184 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471187 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471189 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471193 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471195 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471198 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471201 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471204 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:34:33.472073 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471207 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471209 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471212 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471215 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471218 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471220 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471222 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471225 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471227 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471230 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471232 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471236 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471239 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471241 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471244 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471246 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471249 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471251 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471254 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:34:33.472614 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471256 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471259 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471261 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471264 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471266 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471269 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471271 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471273 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471277 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471280 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471282 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471284 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471287 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471289 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471292 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471295 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471297 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471299 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471302 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.471304 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:34:33.473114 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472027 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472034 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472037 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472039 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472043 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472046 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472049 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472053 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472057 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472060 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472063 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472066 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472070 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472073 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472076 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472078 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472081 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472083 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472086 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472089 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:34:33.473600 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472092 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472095 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472097 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472100 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472102 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472105 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472107 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472109 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472112 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472114 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472117 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472119 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472122 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472124 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472126 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472129 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472131 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472134 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472137 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472139 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:34:33.474116 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472142 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472145 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472147 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472149 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472152 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472155 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472157 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472160 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472162 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472165 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472167 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472170 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472173 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472176 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472179 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472183 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472186 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472189 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472192 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:34:33.474610 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472194 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472197 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472200 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472202 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472205 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472207 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472209 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472211 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472214 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472216 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472218 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472221 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472223 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472226 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472229 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472231 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472234 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472238 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472241 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472244 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:34:33.475092 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472247 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472250 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472252 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472255 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472257 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472261 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472263 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472266 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472269 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472271 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472274 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472276 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472279 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472281 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472283 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472286 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472288 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472290 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472293 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472295 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:34:33.475578 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472680 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472685 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472687 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472690 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472693 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472696 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472699 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472702 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472706 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472709 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472712 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472714 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472717 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472720 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472722 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472725 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472727 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472730 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472732 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472735 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:34:33.476071 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472737 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472740 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472742 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472745 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472749 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472752 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472755 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472758 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472761 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472763 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472766 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472769 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472771 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472774 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472776 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472778 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472781 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472784 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472786 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:34:33.476558 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472789 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472791 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472796 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472819 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472823 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472826 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472829 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472833 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472835 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472838 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472841 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472844 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472846 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472849 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472851 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472854 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472856 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472859 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472861 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472864 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:34:33.477042 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472866 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472868 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472871 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472874 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472876 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472879 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472881 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472883 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472886 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472889 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472891 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472894 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472897 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472901 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472904 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472907 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472909 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472912 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472915 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472918 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:34:33.477572 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472920 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472923 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472925 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472928 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472931 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472935 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472938 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472940 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472943 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472946 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472949 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472951 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472954 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472971 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472974 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472976 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472978 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472981 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472983 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.472986 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:34:33.478074 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473057 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473065 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473071 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473076 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473081 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473085 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473090 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473095 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473098 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473103 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473107 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473111 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473114 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473117 2578 flags.go:64] FLAG: --cgroup-root="" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473120 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473123 2578 flags.go:64] FLAG: --client-ca-file="" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473126 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473129 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473134 2578 flags.go:64] FLAG: --cluster-domain="" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473137 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473141 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473144 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473147 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473151 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 16:34:33.478559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473153 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473157 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473160 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473163 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473165 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473168 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473171 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473174 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473178 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473182 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473184 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473187 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473191 2578 flags.go:64] FLAG: --enable-server="true" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473195 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473200 2578 flags.go:64] FLAG: --event-burst="100" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473203 2578 flags.go:64] FLAG: --event-qps="50" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473205 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473209 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473212 2578 flags.go:64] FLAG: --eviction-hard="" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473215 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473218 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473221 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473224 2578 flags.go:64] FLAG: --eviction-soft="" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473227 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473230 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 16:34:33.479167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473232 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473235 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473238 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473240 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473243 2578 flags.go:64] FLAG: --feature-gates="" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473247 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473249 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473252 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473255 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473258 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473261 2578 flags.go:64] FLAG: --help="false" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473264 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473267 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473270 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473273 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473276 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473279 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473282 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473285 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473287 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473290 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473294 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473297 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473300 2578 flags.go:64] FLAG: --kube-reserved="" Jul 08 16:34:33.479794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473303 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473306 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473309 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473312 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473315 2578 flags.go:64] FLAG: --lock-file="" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473317 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473320 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473323 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473328 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473331 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473334 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473337 2578 flags.go:64] FLAG: --logging-format="text" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473339 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473343 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473345 2578 flags.go:64] FLAG: --manifest-url="" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473348 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473353 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473356 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473360 2578 flags.go:64] FLAG: --max-pods="110" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473363 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473366 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473369 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473372 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473375 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473378 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 16:34:33.480406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473381 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473388 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473391 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473394 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473397 2578 flags.go:64] FLAG: --pod-cidr="" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473400 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473407 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473410 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473413 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473416 2578 flags.go:64] FLAG: --port="10250" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473419 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473422 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05c95268adc1b6086" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473425 2578 flags.go:64] FLAG: --qos-reserved="" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473428 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473431 2578 flags.go:64] FLAG: --register-node="true" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473434 2578 flags.go:64] FLAG: --register-with-taints="" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473437 2578 flags.go:64] FLAG: --registry-burst="10" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473440 2578 flags.go:64] FLAG: --registry-qps="5" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473443 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473445 2578 flags.go:64] FLAG: --reserved-memory="" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473449 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473452 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473456 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473459 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473461 2578 flags.go:64] FLAG: --runonce="false" Jul 08 16:34:33.481067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473464 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473467 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473469 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473472 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473475 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473478 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473481 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473484 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473487 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473489 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473492 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473495 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473498 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473501 2578 flags.go:64] FLAG: --system-cgroups="" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473504 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473509 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473512 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473515 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473519 2578 flags.go:64] FLAG: --tls-min-version="" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473521 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473524 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473526 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473529 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473532 2578 flags.go:64] FLAG: --v="2" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473535 2578 flags.go:64] FLAG: --version="false" Jul 08 16:34:33.481684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473540 2578 flags.go:64] FLAG: --vmodule="" Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473543 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.473546 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473678 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473682 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473685 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473688 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473691 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473693 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473696 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473698 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473700 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473702 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473705 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473707 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473710 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473712 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473716 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473719 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:34:33.482314 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473721 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473724 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473727 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473731 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473734 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473737 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473739 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473742 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473744 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473747 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473749 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473752 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473754 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473756 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473759 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473761 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473764 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473766 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473769 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473771 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:34:33.482786 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473773 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473776 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473778 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473780 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473783 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473785 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473788 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473790 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473792 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473796 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473802 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473805 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473809 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473812 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473816 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473820 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473823 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473826 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473828 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:34:33.483391 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473831 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473834 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473836 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473839 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473841 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473844 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473846 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473848 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473851 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473853 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473856 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473858 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473861 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473863 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473866 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473869 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473871 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473873 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473876 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473878 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:34:33.483860 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473881 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473883 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473886 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473890 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473892 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473895 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473897 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473900 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473903 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473907 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473909 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473912 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473914 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473917 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473919 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473922 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473924 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473927 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473929 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473932 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:34:33.484367 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473934 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473937 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473939 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.473942 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.474512 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.481542 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.481559 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.481582 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.481587 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481644 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481649 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481652 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481656 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481659 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481661 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:34:33.484865 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481664 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481667 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481669 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481672 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481675 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481677 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481681 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481683 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481686 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481689 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481691 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481694 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481696 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481699 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481702 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481704 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481707 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481709 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481712 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481714 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:34:33.485468 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481717 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481719 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481722 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481725 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481728 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481730 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481733 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481736 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481740 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481743 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481746 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481749 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481752 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481754 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481757 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481759 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481761 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481764 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481766 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481771 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:34:33.486140 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481773 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481775 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481778 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481780 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481783 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481786 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481789 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481791 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481794 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481796 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481799 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481802 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481804 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481807 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481809 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481812 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481815 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481818 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481821 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481823 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:34:33.486636 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481826 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481829 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481831 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481834 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481836 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481838 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481841 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481845 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481849 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481852 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481855 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481857 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481861 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481864 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481866 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481869 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481872 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481875 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481878 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:34:33.487238 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481880 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481884 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481887 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481890 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481893 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481896 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481898 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481902 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481905 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481908 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481911 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481914 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481916 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.481919 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.481924 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 16:34:33.487707 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482067 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482072 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482075 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482078 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482081 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482083 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482086 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482089 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482091 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482094 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482096 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482102 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482105 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482107 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482110 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482113 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482116 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482118 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482121 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482123 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:34:33.488107 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482126 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482128 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482131 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482134 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482136 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482139 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482141 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482143 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482146 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482148 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482151 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482153 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482156 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482158 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482161 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482163 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482165 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482168 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482170 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:34:33.488595 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482172 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482175 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482177 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482180 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482182 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482185 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482188 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482192 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482195 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482198 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482200 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482203 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482205 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482207 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482210 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482213 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482215 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482218 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482220 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:34:33.489072 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482222 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482225 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482227 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482230 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482232 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482235 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482237 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482240 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482243 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482246 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482249 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482251 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482254 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482256 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482259 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482262 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482265 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482267 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482270 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:34:33.489535 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482273 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482277 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482280 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482282 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482285 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482287 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482290 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482292 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482294 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482297 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482300 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482302 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482305 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482307 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482310 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482312 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482314 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482317 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482320 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482322 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:34:33.490005 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482325 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:34:33.490539 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:33.482327 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:34:33.490539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.482332 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 16:34:33.490539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.482460 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 16:34:33.490539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.484613 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 16:34:33.490539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.489379 2578 server.go:1019] "Starting client certificate rotation" Jul 08 16:34:33.490539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.489483 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 16:34:33.490539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.489523 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 16:34:33.491903 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.491883 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 16:34:33.493159 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.493142 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 16:34:33.502013 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.501989 2578 log.go:25] "Validated CRI v1 runtime API" Jul 08 16:34:33.504257 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.504239 2578 log.go:25] "Validated CRI v1 image API" Jul 08 16:34:33.505877 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.505862 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 16:34:33.508772 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.508738 2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 83bb0be6-e633-4248-852d-d576d690e3a6:/dev/nvme0n1p3 bb9e515b-06a2-48c2-87e9-93df95315ff8:/dev/nvme0n1p4] Jul 08 16:34:33.508772 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.508768 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 16:34:33.514617 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.514488 2578 manager.go:216] Machine: {Timestamp:2026-07-08 16:34:33.513249625 +0000 UTC m=+0.294489231 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099926 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e2d8d72a479b042f657c434785b54 SystemUUID:ec2e2d8d-72a4-79b0-42f6-57c434785b54 BootID:cbfcbf28-ca82-470a-ab57-91629733a052 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1a:bf:86:ee:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1a:bf:86:ee:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:fb:d1:4c:cb:83 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 16:34:33.514617 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.514597 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 16:34:33.514791 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.514694 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 16:34:33.516827 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.516807 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 16:34:33.517735 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.517706 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 16:34:33.517914 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.517741 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-254.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 16:34:33.518478 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.518465 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 16:34:33.518532 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.518482 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 16:34:33.518532 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.518492 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 16:34:33.519149 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.519137 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 16:34:33.520614 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.520601 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 16:34:33.524789 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.524772 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 08 16:34:33.524985 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.524954 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 16:34:33.525054 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.525014 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 08 16:34:33.525054 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.525030 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 16:34:33.525054 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.525051 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 16:34:33.525177 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.525064 2578 kubelet.go:398] "Adding apiserver pod source" Jul 08 16:34:33.525177 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.525077 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 16:34:33.525492 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.525477 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 16:34:33.525558 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.525503 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 16:34:33.527596 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.527574 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 16:34:33.531016 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.530997 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 16:34:33.531094 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.531031 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 16:34:33.532259 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532245 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 16:34:33.532314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532263 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 16:34:33.532314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532270 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 16:34:33.532314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532280 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 16:34:33.532314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532293 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 16:34:33.532314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532299 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 16:34:33.532314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532307 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 16:34:33.532314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532312 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 16:34:33.532587 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532319 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 16:34:33.532587 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532325 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 16:34:33.532587 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532333 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 16:34:33.532587 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532338 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 16:34:33.532587 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532362 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 16:34:33.532587 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.532368 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 16:34:33.536019 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.536001 2578 server.go:1268] "Started kubelet" Jul 08 16:34:33.536176 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.536133 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 16:34:33.536437 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.536415 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-254.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 16:34:33.536520 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.536410 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 16:34:33.536520 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.536433 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-254.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 16:34:33.536520 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.536125 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 16:34:33.536672 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.536540 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 16:34:33.536948 ip-10-0-130-254 systemd[1]: Started Kubernetes Kubelet. Jul 08 16:34:33.538156 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.538137 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 08 16:34:33.538341 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.538325 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 16:34:33.542331 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.542304 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 16:34:33.544504 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.543674 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-254.ec2.internal.18c05db277e7b151 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-254.ec2.internal,UID:ip-10-0-130-254.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-254.ec2.internal,},FirstTimestamp:2026-07-08 16:34:33.535951185 +0000 UTC m=+0.317190791,LastTimestamp:2026-07-08 16:34:33.535951185 +0000 UTC m=+0.317190791,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-254.ec2.internal,}" Jul 08 16:34:33.545133 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.545118 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 16:34:33.545588 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.545569 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 16:34:33.545690 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.545676 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:33.546196 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546177 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 16:34:33.546196 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546186 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 16:34:33.546358 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546202 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 16:34:33.546358 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546297 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 08 16:34:33.546358 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546304 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 08 16:34:33.546525 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546500 2578 factory.go:55] Registering systemd factory Jul 08 16:34:33.546525 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546513 2578 factory.go:223] Registration of the systemd container factory successfully Jul 08 16:34:33.546768 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546751 2578 factory.go:153] Registering CRI-O factory Jul 08 16:34:33.546815 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546769 2578 factory.go:223] Registration of the crio container factory successfully Jul 08 16:34:33.546851 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546824 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 16:34:33.546851 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546847 2578 factory.go:103] Registering Raw factory Jul 08 16:34:33.546928 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.546861 2578 manager.go:1196] Started watching for new ooms in manager Jul 08 16:34:33.547188 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.547160 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-254.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 16:34:33.547260 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.547211 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 16:34:33.547316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.547298 2578 manager.go:349] Starting recovery of all containers Jul 08 16:34:33.553083 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.552859 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-98m5m" Jul 08 16:34:33.559831 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.559806 2578 manager.go:354] Recovery completed Jul 08 16:34:33.559831 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.559789 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-98m5m" Jul 08 16:34:33.565288 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.565271 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:34:33.568533 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.568514 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:34:33.568602 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.568547 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:34:33.568602 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.568561 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:34:33.569157 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.569144 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 16:34:33.569157 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.569157 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 16:34:33.569281 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.569159 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:34:33.569281 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.569175 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 08 16:34:33.571300 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.571286 2578 policy_none.go:49] "None policy: Start" Jul 08 16:34:33.571364 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.571306 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 16:34:33.571364 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.571320 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 16:34:33.572426 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.572402 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:34:33.572492 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.572438 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:34:33.572492 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.572454 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:34:33.573438 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.573427 2578 policy_none.go:47] "Start" Jul 08 16:34:33.613615 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.613592 2578 manager.go:337] "Starting Device Plugin manager" Jul 08 16:34:33.613785 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.613631 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 16:34:33.613785 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.613641 2578 server.go:87] "Starting device plugin registration server" Jul 08 16:34:33.613973 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.613947 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 16:34:33.614090 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.613989 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 16:34:33.614143 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.614085 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 16:34:33.614191 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.614166 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 16:34:33.614191 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.614174 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 16:34:33.615774 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.615742 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 16:34:33.615861 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.615806 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:33.679950 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.679903 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 16:34:33.682244 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.681273 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 16:34:33.682244 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.681317 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 16:34:33.682244 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.681341 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 16:34:33.682244 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.681382 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 16:34:33.683766 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.683744 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:34:33.714914 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.714817 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:34:33.716218 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.716200 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:34:33.716286 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.716234 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:34:33.716286 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.716245 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:34:33.716286 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.716281 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.723793 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.723776 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.723858 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.723804 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-254.ec2.internal\": node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:33.735155 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.735127 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:33.782404 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.782363 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal"] Jul 08 16:34:33.782511 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.782449 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:34:33.784151 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.784131 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:34:33.784246 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.784163 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:34:33.784246 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.784173 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:34:33.786565 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.786550 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:34:33.786679 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.786664 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.786716 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.786694 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:34:33.787369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.787352 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:34:33.787450 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.787383 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:34:33.787450 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.787396 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:34:33.787992 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.787976 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:34:33.788058 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.788006 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:34:33.788058 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.788016 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:34:33.790231 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.790215 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.790312 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.790240 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:34:33.791020 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.791001 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:34:33.791094 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.791035 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:34:33.791094 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.791058 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:34:33.815780 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.815747 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-254.ec2.internal\" not found" node="ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.820065 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.820045 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-254.ec2.internal\" not found" node="ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.836239 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.836215 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:33.848183 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.848158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08c4ac20e025659ba7da192211a60491-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal\" (UID: \"08c4ac20e025659ba7da192211a60491\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.848252 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.848191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1b906b0da08878c33265f8e1596ed7a5-config\") pod \"kube-apiserver-proxy-ip-10-0-130-254.ec2.internal\" (UID: \"1b906b0da08878c33265f8e1596ed7a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.848252 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.848215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/08c4ac20e025659ba7da192211a60491-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal\" (UID: \"08c4ac20e025659ba7da192211a60491\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.936784 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:33.936733 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:33.949138 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.949109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/08c4ac20e025659ba7da192211a60491-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal\" (UID: \"08c4ac20e025659ba7da192211a60491\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.949138 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.949141 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08c4ac20e025659ba7da192211a60491-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal\" (UID: \"08c4ac20e025659ba7da192211a60491\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.949325 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.949166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1b906b0da08878c33265f8e1596ed7a5-config\") pod \"kube-apiserver-proxy-ip-10-0-130-254.ec2.internal\" (UID: \"1b906b0da08878c33265f8e1596ed7a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.949325 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.949219 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/08c4ac20e025659ba7da192211a60491-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal\" (UID: \"08c4ac20e025659ba7da192211a60491\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.949325 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.949219 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1b906b0da08878c33265f8e1596ed7a5-config\") pod \"kube-apiserver-proxy-ip-10-0-130-254.ec2.internal\" (UID: \"1b906b0da08878c33265f8e1596ed7a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" Jul 08 16:34:33.949325 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:33.949224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08c4ac20e025659ba7da192211a60491-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal\" (UID: \"08c4ac20e025659ba7da192211a60491\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:35.091920 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.091885 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.174454 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:35.174406 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:35.178114 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:35.178080 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" Jul 08 16:34:35.192150 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.192119 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.292768 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.292712 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.393318 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.393265 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.493922 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.493868 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.543357 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:35.543330 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 16:34:35.543940 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:35.543489 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 16:34:35.594587 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.594546 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.617369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:35.617324 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 16:29:34 +0000 UTC" deadline="2027-12-02 08:06:48.393619274 +0000 UTC" Jul 08 16:34:35.617369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:35.617361 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12279h32m12.776261351s" Jul 08 16:34:35.695706 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.695629 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.796378 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.796344 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.830218 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:35.830179 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b906b0da08878c33265f8e1596ed7a5.slice/crio-e4972769d174a89e9b467a826ee8706200e2bae0f86a1e891a47a83a027fc4cd WatchSource:0}: Error finding container e4972769d174a89e9b467a826ee8706200e2bae0f86a1e891a47a83a027fc4cd: Status 404 returned error can't find the container with id e4972769d174a89e9b467a826ee8706200e2bae0f86a1e891a47a83a027fc4cd Jul 08 16:34:35.830933 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:35.830909 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c4ac20e025659ba7da192211a60491.slice/crio-f7e0a244e0bc621b05b2f77769d32bd01604244ebfb2262416c7339f98090d08 WatchSource:0}: Error finding container f7e0a244e0bc621b05b2f77769d32bd01604244ebfb2262416c7339f98090d08: Status 404 returned error can't find the container with id f7e0a244e0bc621b05b2f77769d32bd01604244ebfb2262416c7339f98090d08 Jul 08 16:34:35.837369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:35.837352 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 16:34:35.896778 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:35.896738 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-254.ec2.internal\" not found" Jul 08 16:34:35.935581 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:35.935549 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:34:36.000733 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.000640 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" Jul 08 16:34:36.013758 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.013735 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 16:34:36.015537 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.015519 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" Jul 08 16:34:36.022935 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.022910 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:34:36.024597 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.024548 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 16:34:36.144323 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.144293 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:34:36.478864 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.478832 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:34:36.580739 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.580648 2578 apiserver.go:52] "Watching apiserver" Jul 08 16:34:36.589724 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.589691 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 16:34:36.590870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.590846 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-xcqgq","kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9","openshift-dns/node-resolver-pqjtx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal","openshift-multus/multus-additional-cni-plugins-5bnw6","openshift-multus/network-metrics-daemon-9gh26","openshift-network-diagnostics/network-check-target-knbpm","openshift-network-operator/iptables-alerter-gwjxc","kube-system/global-pull-secret-syncer-zbrcz","kube-system/konnectivity-agent-s6xcp","openshift-cluster-node-tuning-operator/tuned-pthc2","openshift-image-registry/node-ca-22m2d","openshift-multus/multus-lnq7s"] Jul 08 16:34:36.593391 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.593364 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:34:36.595427 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.595405 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 16:34:36.595539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.595494 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 16:34:36.595987 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.595696 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jrnql\"" Jul 08 16:34:36.595987 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.595713 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.597327 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.597303 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 16:34:36.597889 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.597713 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 16:34:36.597889 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.597770 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 16:34:36.597889 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.597795 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wg2vj\"" Jul 08 16:34:36.597889 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.597874 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.599820 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.599564 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 16:34:36.599820 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.599740 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 16:34:36.599974 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.599906 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 16:34:36.599974 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.599920 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-r9kqb\"" Jul 08 16:34:36.600149 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.600133 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.600366 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.600347 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 16:34:36.602311 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.602257 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qztvz\"" Jul 08 16:34:36.602406 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.602325 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 16:34:36.602464 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.602409 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.602741 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.602723 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 16:34:36.604432 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.604415 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rz62j\"" Jul 08 16:34:36.604780 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.604761 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.604870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.604782 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 16:34:36.604870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.604810 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 16:34:36.607052 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.607036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:36.607168 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.607140 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:36.609194 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.609168 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 16:34:36.609436 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.609418 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 16:34:36.609510 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.609435 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gpjbp\"" Jul 08 16:34:36.609854 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.609722 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 16:34:36.609854 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.609736 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 16:34:36.609854 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.609808 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:36.609854 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.609824 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 16:34:36.610106 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.609873 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:36.611550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611522 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-device-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.611649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611560 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-hosts-file\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.611649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611585 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-run\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.611649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-modprobe-d\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.611649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysctl-conf\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-tuned\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-sys-fs\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-sys\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611745 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-tmp-dir\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-lib-modules\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-socket-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611835 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-registration-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.611870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-etc-selinux\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxwl6\" (UniqueName: \"kubernetes.io/projected/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-kube-api-access-qxwl6\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611908 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4jgt\" (UniqueName: \"kubernetes.io/projected/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-kube-api-access-q4jgt\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg76d\" (UniqueName: \"kubernetes.io/projected/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-kube-api-access-kg76d\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.611995 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysconfig\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysctl-d\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-systemd\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612064 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/864a8a9f-7cae-43e4-9736-2174d574a7e8-agent-certs\") pod \"konnectivity-agent-s6xcp\" (UID: \"864a8a9f-7cae-43e4-9736-2174d574a7e8\") " pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-host\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-kubernetes\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612139 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-tmp\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/864a8a9f-7cae-43e4-9736-2174d574a7e8-konnectivity-ca\") pod \"konnectivity-agent-s6xcp\" (UID: \"864a8a9f-7cae-43e4-9736-2174d574a7e8\") " pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612184 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-serviceca\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612207 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c29vz\" (UniqueName: \"kubernetes.io/projected/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-kube-api-access-c29vz\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-var-lib-kubelet\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.612316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-host\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.613007 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.612524 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.614151 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.614072 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.614249 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.614175 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:36.614495 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.614467 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 16:34:36.614584 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.614540 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 16:34:36.614789 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.614774 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vvnmm\"" Jul 08 16:34:36.614847 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.614795 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 16:34:36.616532 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.616512 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 16:34:36.616784 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.616767 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.618461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.618418 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mg2dt\"" Jul 08 16:34:36.618598 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.618582 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 16:34:36.619064 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.618902 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 16:34:36.619064 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.618999 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 16:34:36.619064 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.619003 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 16:34:36.622298 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.619814 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.622298 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.619214 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 16:34:36.622298 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.620156 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 16:34:36.622298 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.621924 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 16:34:36.622298 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.622228 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lqszb\"" Jul 08 16:34:36.637630 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.637582 2578 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1" Jul 08 16:34:36.637745 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.637713 2578 kuberuntime_manager.go:1459] "Unhandled Error" err="container haproxy start failed in pod kube-apiserver-proxy-ip-10-0-130-254.ec2.internal_kube-system(1b906b0da08878c33265f8e1596ed7a5): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 08 16:34:36.637798 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.637753 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"haproxy\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" podUID="1b906b0da08878c33265f8e1596ed7a5" Jul 08 16:34:36.701012 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.700976 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x2mxr" Jul 08 16:34:36.701840 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.701815 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 16:34:36.712699 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-ovnkube-script-lib\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.712861 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712714 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-cni-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.712861 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lftmt\" (UniqueName: \"kubernetes.io/projected/e10fb928-0f80-4c9e-a510-297440e94a5f-kube-api-access-lftmt\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.712861 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cnibin\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.712861 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/864a8a9f-7cae-43e4-9736-2174d574a7e8-agent-certs\") pod \"konnectivity-agent-s6xcp\" (UID: \"864a8a9f-7cae-43e4-9736-2174d574a7e8\") " pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:34:36.712861 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-kubernetes\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.713059 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-tmp\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.713059 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712925 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-kubelet\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.713059 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712934 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-kubernetes\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.713059 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-multus-certs\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.713059 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.712973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-os-release\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.713059 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/864a8a9f-7cae-43e4-9736-2174d574a7e8-konnectivity-ca\") pod \"konnectivity-agent-s6xcp\" (UID: \"864a8a9f-7cae-43e4-9736-2174d574a7e8\") " pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:34:36.713059 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713054 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-serviceca\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.713448 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c29vz\" (UniqueName: \"kubernetes.io/projected/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-kube-api-access-c29vz\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.713448 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713102 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-host\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.713448 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:36.713448 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713198 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-host\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.713448 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713196 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-os-release\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.713448 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cni-binary-copy\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.713448 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713236 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 16:34:36.713448 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-device-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.713833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713598 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-device-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.713833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-run\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.713833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-ovn\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.713833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713735 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-run\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.713833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713736 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-serviceca\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.713833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713751 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/864a8a9f-7cae-43e4-9736-2174d574a7e8-konnectivity-ca\") pod \"konnectivity-agent-s6xcp\" (UID: \"864a8a9f-7cae-43e4-9736-2174d574a7e8\") " pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:34:36.713833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysctl-conf\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.713833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713809 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-tuned\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713836 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-cni-bin\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-cni-netd\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713881 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-netns\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713887 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysctl-conf\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713905 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16660cb9-13e7-489c-bad0-08851d9f10b3-host-slash\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-kubelet\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.713996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgp9z\" (UniqueName: \"kubernetes.io/projected/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-kube-api-access-wgp9z\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e10fb928-0f80-4c9e-a510-297440e94a5f-cni-binary-copy\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714141 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.714266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-k8s-cni-cncf-io\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-hostroot\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714391 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16660cb9-13e7-489c-bad0-08851d9f10b3-iptables-alerter-script\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-registration-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-etc-selinux\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cdda881-c366-4e95-ab7e-6491a2e4c00b-kubelet-config\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-etc-kubernetes\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg76d\" (UniqueName: \"kubernetes.io/projected/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-kube-api-access-kg76d\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysctl-d\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-systemd\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-var-lib-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714668 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-etc-selinux\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-node-log\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-ovnkube-config\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714729 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-systemd\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.714757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714686 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-registration-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55p74\" (UniqueName: \"kubernetes.io/projected/823540fd-8449-4516-ab31-489f22b48084-kube-api-access-55p74\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysctl-d\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-daemon-config\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-host\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714933 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-cni-multus\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.714988 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-host\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715021 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-var-lib-kubelet\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cdda881-c366-4e95-ab7e-6491a2e4c00b-dbus\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q74p\" (UniqueName: \"kubernetes.io/projected/7109969a-f717-4285-b47f-261f6b3353a2-kube-api-access-7q74p\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715077 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-var-lib-kubelet\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-hosts-file\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715146 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-hosts-file\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-run-ovn-kubernetes\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-conf-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-modprobe-d\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-slash\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.715462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-run-netns\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-log-socket\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-system-cni-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-cni-bin\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-sys-fs\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-sys\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-etc-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715524 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-socket-dir-parent\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-tmp-dir\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-lib-modules\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-system-cni-dir\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-socket-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715738 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxwl6\" (UniqueName: \"kubernetes.io/projected/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-kube-api-access-qxwl6\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4jgt\" (UniqueName: \"kubernetes.io/projected/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-kube-api-access-q4jgt\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715790 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-systemd\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-env-overrides\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.715958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715838 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x2mxr" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-cnibin\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715886 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zkrx\" (UniqueName: \"kubernetes.io/projected/16660cb9-13e7-489c-bad0-08851d9f10b3-kube-api-access-4zkrx\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysconfig\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-systemd-units\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.715970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/823540fd-8449-4516-ab31-489f22b48084-ovn-node-metrics-cert\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716220 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-lib-modules\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716355 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-socket-dir\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716416 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-tmp-dir\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.716488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716485 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-modprobe-d\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.716881 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716530 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-sys-fs\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.716881 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716571 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-sys\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.716881 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716659 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-sysconfig\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.717006 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.716987 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-etc-tuned\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.717505 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.717485 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/864a8a9f-7cae-43e4-9736-2174d574a7e8-agent-certs\") pod \"konnectivity-agent-s6xcp\" (UID: \"864a8a9f-7cae-43e4-9736-2174d574a7e8\") " pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:34:36.717729 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.717667 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-tmp\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.724767 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.724727 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c29vz\" (UniqueName: \"kubernetes.io/projected/ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b-kube-api-access-c29vz\") pod \"aws-ebs-csi-driver-node-chnr9\" (UID: \"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.725927 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.725904 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg76d\" (UniqueName: \"kubernetes.io/projected/c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc-kube-api-access-kg76d\") pod \"node-ca-22m2d\" (UID: \"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc\") " pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.726032 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.725924 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxwl6\" (UniqueName: \"kubernetes.io/projected/ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a-kube-api-access-qxwl6\") pod \"node-resolver-pqjtx\" (UID: \"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a\") " pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.726439 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.726398 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4jgt\" (UniqueName: \"kubernetes.io/projected/65e2fcb0-31cc-456c-a4e9-4896bfa4cc41-kube-api-access-q4jgt\") pod \"tuned-pthc2\" (UID: \"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41\") " pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.740603 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.740547 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" event={"ID":"08c4ac20e025659ba7da192211a60491","Type":"ContainerStarted","Data":"f7e0a244e0bc621b05b2f77769d32bd01604244ebfb2262416c7339f98090d08"} Jul 08 16:34:36.741769 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.741745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" event={"ID":"1b906b0da08878c33265f8e1596ed7a5","Type":"ContainerStarted","Data":"e4972769d174a89e9b467a826ee8706200e2bae0f86a1e891a47a83a027fc4cd"} Jul 08 16:34:36.742784 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.742750 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"haproxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" podUID="1b906b0da08878c33265f8e1596ed7a5" Jul 08 16:34:36.816766 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e10fb928-0f80-4c9e-a510-297440e94a5f-cni-binary-copy\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.816766 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-k8s-cni-cncf-io\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-hostroot\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16660cb9-13e7-489c-bad0-08851d9f10b3-iptables-alerter-script\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cdda881-c366-4e95-ab7e-6491a2e4c00b-kubelet-config\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816857 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816869 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-hostroot\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816870 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-k8s-cni-cncf-io\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-etc-kubernetes\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.816959 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-var-lib-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.816979 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cdda881-c366-4e95-ab7e-6491a2e4c00b-kubelet-config\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.816999 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-node-log\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.817024 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret podName:1cdda881-c366-4e95-ab7e-6491a2e4c00b nodeName:}" failed. No retries permitted until 2026-07-08 16:34:37.317002072 +0000 UTC m=+3.043926952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret") pod "global-pull-secret-syncer-zbrcz" (UID: "1cdda881-c366-4e95-ab7e-6491a2e4c00b") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817042 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-node-log\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817044 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-etc-kubernetes\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817078 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-ovnkube-config\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-var-lib-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817170 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55p74\" (UniqueName: \"kubernetes.io/projected/823540fd-8449-4516-ab31-489f22b48084-kube-api-access-55p74\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817224 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-daemon-config\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817253 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-cni-multus\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cdda881-c366-4e95-ab7e-6491a2e4c00b-dbus\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q74p\" (UniqueName: \"kubernetes.io/projected/7109969a-f717-4285-b47f-261f6b3353a2-kube-api-access-7q74p\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817338 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-cni-multus\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-run-ovn-kubernetes\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-conf-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817388 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-run-ovn-kubernetes\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-slash\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817435 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e10fb928-0f80-4c9e-a510-297440e94a5f-cni-binary-copy\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.817639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-run-netns\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-log-socket\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817495 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-conf-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-system-cni-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817499 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16660cb9-13e7-489c-bad0-08851d9f10b3-iptables-alerter-script\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-cni-bin\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817529 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cdda881-c366-4e95-ab7e-6491a2e4c00b-dbus\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817546 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-slash\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-etc-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817578 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-cni-bin\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817583 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-run-netns\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817589 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-system-cni-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-socket-dir-parent\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817607 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-ovnkube-config\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817626 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-log-socket\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817630 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-system-cni-dir\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817640 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-etc-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817643 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-socket-dir-parent\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.818424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817665 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-system-cni-dir\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817685 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-daemon-config\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-systemd\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-env-overrides\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-cnibin\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zkrx\" (UniqueName: \"kubernetes.io/projected/16660cb9-13e7-489c-bad0-08851d9f10b3-kube-api-access-4zkrx\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-systemd\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-systemd-units\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817804 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-cnibin\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817806 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817842 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-systemd-units\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-openvswitch\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/823540fd-8449-4516-ab31-489f22b48084-ovn-node-metrics-cert\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-ovnkube-script-lib\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-cni-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lftmt\" (UniqueName: \"kubernetes.io/projected/e10fb928-0f80-4c9e-a510-297440e94a5f-kube-api-access-lftmt\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.819433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cnibin\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817989 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-multus-cni-dir\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.817999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-kubelet\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-multus-certs\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818033 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cnibin\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-os-release\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818074 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-var-lib-kubelet\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818109 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-env-overrides\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818114 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-multus-certs\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818130 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-os-release\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818160 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cni-binary-copy\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818175 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-os-release\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-ovn\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818240 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-run-ovn\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.818243 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818245 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-os-release\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.820249 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-cni-bin\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.818315 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs podName:7109969a-f717-4285-b47f-261f6b3353a2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:37.318298041 +0000 UTC m=+3.045222895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs") pod "network-metrics-daemon-9gh26" (UID: "7109969a-f717-4285-b47f-261f6b3353a2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818328 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-cni-bin\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-cni-netd\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-netns\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16660cb9-13e7-489c-bad0-08851d9f10b3-host-slash\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818421 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-cni-netd\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818425 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/823540fd-8449-4516-ab31-489f22b48084-ovnkube-script-lib\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-kubelet\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818454 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e10fb928-0f80-4c9e-a510-297440e94a5f-host-run-netns\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818510 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-kubelet\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgp9z\" (UniqueName: \"kubernetes.io/projected/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-kube-api-access-wgp9z\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818557 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823540fd-8449-4516-ab31-489f22b48084-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.821050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818657 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16660cb9-13e7-489c-bad0-08851d9f10b3-host-slash\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.821717 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818811 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cni-binary-copy\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.821717 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.818883 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.821717 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.819047 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.821717 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.820827 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/823540fd-8449-4516-ab31-489f22b48084-ovn-node-metrics-cert\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.823856 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.823833 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:34:36.823856 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.823859 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:34:36.824043 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.823875 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vnjbc for pod openshift-network-diagnostics/network-check-target-knbpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:36.824043 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:36.823928 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc podName:2812fd9f-259d-44ae-91fa-d839fe4b94b9 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:37.323916317 +0000 UTC m=+3.050841168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vnjbc" (UniqueName: "kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc") pod "network-check-target-knbpm" (UID: "2812fd9f-259d-44ae-91fa-d839fe4b94b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:36.829573 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.829522 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zkrx\" (UniqueName: \"kubernetes.io/projected/16660cb9-13e7-489c-bad0-08851d9f10b3-kube-api-access-4zkrx\") pod \"iptables-alerter-gwjxc\" (UID: \"16660cb9-13e7-489c-bad0-08851d9f10b3\") " pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.829713 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.829670 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgp9z\" (UniqueName: \"kubernetes.io/projected/0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61-kube-api-access-wgp9z\") pod \"multus-additional-cni-plugins-5bnw6\" (UID: \"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61\") " pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.829774 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.829677 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lftmt\" (UniqueName: \"kubernetes.io/projected/e10fb928-0f80-4c9e-a510-297440e94a5f-kube-api-access-lftmt\") pod \"multus-lnq7s\" (UID: \"e10fb928-0f80-4c9e-a510-297440e94a5f\") " pod="openshift-multus/multus-lnq7s" Jul 08 16:34:36.829834 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.829783 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q74p\" (UniqueName: \"kubernetes.io/projected/7109969a-f717-4285-b47f-261f6b3353a2-kube-api-access-7q74p\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:36.829924 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.829904 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55p74\" (UniqueName: \"kubernetes.io/projected/823540fd-8449-4516-ab31-489f22b48084-kube-api-access-55p74\") pod \"ovnkube-node-xcqgq\" (UID: \"823540fd-8449-4516-ab31-489f22b48084\") " pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.904886 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.904844 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:34:36.913218 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.913194 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" Jul 08 16:34:36.923125 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.923100 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-22m2d" Jul 08 16:34:36.930900 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.930876 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pqjtx" Jul 08 16:34:36.937747 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.937714 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pthc2" Jul 08 16:34:36.944522 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.944496 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" Jul 08 16:34:36.952265 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.952244 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gwjxc" Jul 08 16:34:36.959985 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.959961 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:34:36.963760 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:36.963726 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lnq7s" Jul 08 16:34:37.186616 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:37.186591 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6a0a0e_0dd4_42f0_ad7f_3d044cf5d35a.slice/crio-08df763183229aae6288a6f9d17b174122dd4e4ef5bd95c358a28e245c3bd9a1 WatchSource:0}: Error finding container 08df763183229aae6288a6f9d17b174122dd4e4ef5bd95c358a28e245c3bd9a1: Status 404 returned error can't find the container with id 08df763183229aae6288a6f9d17b174122dd4e4ef5bd95c358a28e245c3bd9a1 Jul 08 16:34:37.201297 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:34:37.201128 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc854cc_e69e_4cbb_ac2f_37f52a0d8d61.slice/crio-1a28cba51f37395239c73a3fb450a5e9e032d70d662d81da578f3e07c4555118 WatchSource:0}: Error finding container 1a28cba51f37395239c73a3fb450a5e9e032d70d662d81da578f3e07c4555118: Status 404 returned error can't find the container with id 1a28cba51f37395239c73a3fb450a5e9e032d70d662d81da578f3e07c4555118 Jul 08 16:34:37.322717 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.322534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:37.322872 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.322751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:37.322872 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.322673 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:37.322872 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.322828 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs podName:7109969a-f717-4285-b47f-261f6b3353a2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:38.322811363 +0000 UTC m=+4.049736214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs") pod "network-metrics-daemon-9gh26" (UID: "7109969a-f717-4285-b47f-261f6b3353a2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:37.323027 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.322876 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:37.323027 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.322935 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret podName:1cdda881-c366-4e95-ab7e-6491a2e4c00b nodeName:}" failed. No retries permitted until 2026-07-08 16:34:38.322919191 +0000 UTC m=+4.049844049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret") pod "global-pull-secret-syncer-zbrcz" (UID: "1cdda881-c366-4e95-ab7e-6491a2e4c00b") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:37.423340 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.423309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:37.423507 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.423469 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:34:37.423507 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.423487 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:34:37.423507 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.423498 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vnjbc for pod openshift-network-diagnostics/network-check-target-knbpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:37.423630 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.423546 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc podName:2812fd9f-259d-44ae-91fa-d839fe4b94b9 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:38.423533997 +0000 UTC m=+4.150458849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vnjbc" (UniqueName: "kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc") pod "network-check-target-knbpm" (UID: "2812fd9f-259d-44ae-91fa-d839fe4b94b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:37.731639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.731550 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 16:29:36 +0000 UTC" deadline="2028-01-05 17:14:40.630730569 +0000 UTC" Jul 08 16:34:37.731639 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.731592 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13104h40m2.899143408s" Jul 08 16:34:37.748487 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.748444 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lnq7s" event={"ID":"e10fb928-0f80-4c9e-a510-297440e94a5f","Type":"ContainerStarted","Data":"38bde52a6e34e32276ab2280229911c92de2b45625d80eb8789133779d3ee603"} Jul 08 16:34:37.750830 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.750797 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" event={"ID":"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b","Type":"ContainerStarted","Data":"618c4d63e6da3b71ede0d49178d896acf75aa6ce63e130af69901a867b92daa3"} Jul 08 16:34:37.752979 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.752945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" event={"ID":"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61","Type":"ContainerStarted","Data":"1a28cba51f37395239c73a3fb450a5e9e032d70d662d81da578f3e07c4555118"} Jul 08 16:34:37.760186 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.760146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pqjtx" event={"ID":"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a","Type":"ContainerStarted","Data":"08df763183229aae6288a6f9d17b174122dd4e4ef5bd95c358a28e245c3bd9a1"} Jul 08 16:34:37.766755 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.766457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"d88b64ee915a52f031dece516827bbc887e00faae4060051cf98be363da14c69"} Jul 08 16:34:37.768845 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.768812 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s6xcp" event={"ID":"864a8a9f-7cae-43e4-9736-2174d574a7e8","Type":"ContainerStarted","Data":"4a0279c743d3d71a90ac5f629f661169e386f9092368c5759088018570677a27"} Jul 08 16:34:37.779217 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.779167 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gwjxc" event={"ID":"16660cb9-13e7-489c-bad0-08851d9f10b3","Type":"ContainerStarted","Data":"afb634c0a0ab841061aa0c4c7fc6c92729e161ba1e419df59ac6172f2647939c"} Jul 08 16:34:37.796648 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.796600 2578 generic.go:356] "Generic (PLEG): container finished" podID="08c4ac20e025659ba7da192211a60491" containerID="eeeaad1c4784b8c410312b4565bdde6c594f207b3b1ba471b97b9bafb01ccc74" exitCode=0 Jul 08 16:34:37.796903 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.796873 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" event={"ID":"08c4ac20e025659ba7da192211a60491","Type":"ContainerDied","Data":"eeeaad1c4784b8c410312b4565bdde6c594f207b3b1ba471b97b9bafb01ccc74"} Jul 08 16:34:37.801608 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.801504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pthc2" event={"ID":"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41","Type":"ContainerStarted","Data":"3d27c07ffe74f7f049619cbee54dff0aae4dc8ad99f511b1e40ee6507d293a97"} Jul 08 16:34:37.809463 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:37.809429 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-22m2d" event={"ID":"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc","Type":"ContainerStarted","Data":"eca43dca9d4e0bd63cff2fffb49b482d976150c502d9b463850834c59558089a"} Jul 08 16:34:37.827788 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:37.827744 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"haproxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" podUID="1b906b0da08878c33265f8e1596ed7a5" Jul 08 16:34:38.331560 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.331456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:38.331560 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.331530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:38.331797 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.331661 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:38.331797 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.331728 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret podName:1cdda881-c366-4e95-ab7e-6491a2e4c00b nodeName:}" failed. No retries permitted until 2026-07-08 16:34:40.33170826 +0000 UTC m=+6.058633117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret") pod "global-pull-secret-syncer-zbrcz" (UID: "1cdda881-c366-4e95-ab7e-6491a2e4c00b") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:38.332139 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.332031 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:38.332139 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.332100 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs podName:7109969a-f717-4285-b47f-261f6b3353a2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:40.332081456 +0000 UTC m=+6.059006320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs") pod "network-metrics-daemon-9gh26" (UID: "7109969a-f717-4285-b47f-261f6b3353a2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:38.432688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.432645 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:38.432892 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.432866 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:34:38.432959 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.432888 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:34:38.432959 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.432909 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vnjbc for pod openshift-network-diagnostics/network-check-target-knbpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:38.433042 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.432973 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc podName:2812fd9f-259d-44ae-91fa-d839fe4b94b9 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:40.432954092 +0000 UTC m=+6.159878945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vnjbc" (UniqueName: "kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc") pod "network-check-target-knbpm" (UID: "2812fd9f-259d-44ae-91fa-d839fe4b94b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:38.732650 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.732550 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 16:29:36 +0000 UTC" deadline="2028-04-05 13:03:11.460650616 +0000 UTC" Jul 08 16:34:38.732650 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.732613 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15284h28m32.728041898s" Jul 08 16:34:38.738013 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.737449 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:38.738013 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.737448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:38.738013 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.737667 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:38.738013 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.737789 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:38.738013 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.737847 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:38.738013 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:38.737920 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:38.826073 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:38.825939 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" event={"ID":"08c4ac20e025659ba7da192211a60491","Type":"ContainerStarted","Data":"29cbd699ef22391c3aab6cb94d8838f6a4af34098576b5d7620f6a6d511ef70d"} Jul 08 16:34:40.349899 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:40.348326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:40.349899 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:40.348440 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:40.349899 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.348582 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:40.349899 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.348649 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs podName:7109969a-f717-4285-b47f-261f6b3353a2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:44.348630203 +0000 UTC m=+10.075555063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs") pod "network-metrics-daemon-9gh26" (UID: "7109969a-f717-4285-b47f-261f6b3353a2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:40.349899 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.349107 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:40.349899 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.349164 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret podName:1cdda881-c366-4e95-ab7e-6491a2e4c00b nodeName:}" failed. No retries permitted until 2026-07-08 16:34:44.349147393 +0000 UTC m=+10.076072247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret") pod "global-pull-secret-syncer-zbrcz" (UID: "1cdda881-c366-4e95-ab7e-6491a2e4c00b") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:40.449675 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:40.449624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:40.449865 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.449821 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:34:40.449865 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.449851 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:34:40.449865 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.449866 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vnjbc for pod openshift-network-diagnostics/network-check-target-knbpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:40.450018 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.449930 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc podName:2812fd9f-259d-44ae-91fa-d839fe4b94b9 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:44.449909497 +0000 UTC m=+10.176834355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vnjbc" (UniqueName: "kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc") pod "network-check-target-knbpm" (UID: "2812fd9f-259d-44ae-91fa-d839fe4b94b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:40.737423 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:40.737388 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:40.737607 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.737533 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:40.737967 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:40.737948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:40.738086 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.738065 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:40.738210 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:40.738192 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:40.738326 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:40.738296 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:42.736035 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:42.736004 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:42.736535 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:42.736138 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:42.736606 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:42.736537 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:42.736714 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:42.736685 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:42.736789 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:42.736750 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:42.736849 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:42.736830 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:44.380465 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:44.380426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:44.380933 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:44.380484 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:44.380933 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.380586 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:44.380933 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.380585 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:44.380933 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.380650 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs podName:7109969a-f717-4285-b47f-261f6b3353a2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:52.380634337 +0000 UTC m=+18.107559189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs") pod "network-metrics-daemon-9gh26" (UID: "7109969a-f717-4285-b47f-261f6b3353a2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:44.380933 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.380665 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret podName:1cdda881-c366-4e95-ab7e-6491a2e4c00b nodeName:}" failed. No retries permitted until 2026-07-08 16:34:52.380659051 +0000 UTC m=+18.107583902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret") pod "global-pull-secret-syncer-zbrcz" (UID: "1cdda881-c366-4e95-ab7e-6491a2e4c00b") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:44.481818 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:44.481788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:44.482010 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.481973 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:34:44.482010 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.481996 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:34:44.482010 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.482008 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vnjbc for pod openshift-network-diagnostics/network-check-target-knbpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:44.482162 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.482068 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc podName:2812fd9f-259d-44ae-91fa-d839fe4b94b9 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:52.4820497 +0000 UTC m=+18.208974550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vnjbc" (UniqueName: "kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc") pod "network-check-target-knbpm" (UID: "2812fd9f-259d-44ae-91fa-d839fe4b94b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:44.737537 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:44.737491 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:44.737720 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.737622 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:44.738016 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:44.738000 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:44.738107 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.738090 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:44.738313 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:44.738265 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:44.738416 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:44.738394 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:46.736690 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:46.736662 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:46.736690 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:46.736691 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:46.737128 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:46.736666 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:46.737128 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:46.736777 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:46.737128 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:46.736869 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:46.737128 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:46.736915 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:48.736399 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:48.736360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:48.736894 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:48.736498 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:48.736969 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:48.736918 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:48.737060 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:48.737029 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:48.737270 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:48.737108 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:48.737270 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:48.737174 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:50.736324 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:50.736257 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:50.736324 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:50.736307 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:50.736810 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:50.736258 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:50.736810 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:50.736423 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:50.736810 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:50.736492 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:50.736810 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:50.736567 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:52.443541 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:52.443499 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:52.444048 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:52.443566 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:52.444048 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.443652 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:52.444048 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.443663 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:52.444048 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.443723 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs podName:7109969a-f717-4285-b47f-261f6b3353a2 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:08.443706116 +0000 UTC m=+34.170630966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs") pod "network-metrics-daemon-9gh26" (UID: "7109969a-f717-4285-b47f-261f6b3353a2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:34:52.444048 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.443741 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret podName:1cdda881-c366-4e95-ab7e-6491a2e4c00b nodeName:}" failed. No retries permitted until 2026-07-08 16:35:08.443731687 +0000 UTC m=+34.170656537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret") pod "global-pull-secret-syncer-zbrcz" (UID: "1cdda881-c366-4e95-ab7e-6491a2e4c00b") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:34:52.545071 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:52.545034 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:52.545303 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.545192 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:34:52.545303 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.545213 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:34:52.545303 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.545224 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vnjbc for pod openshift-network-diagnostics/network-check-target-knbpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:52.545493 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.545310 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc podName:2812fd9f-259d-44ae-91fa-d839fe4b94b9 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:08.545291446 +0000 UTC m=+34.272216311 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vnjbc" (UniqueName: "kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc") pod "network-check-target-knbpm" (UID: "2812fd9f-259d-44ae-91fa-d839fe4b94b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:34:52.737051 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:52.736964 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:52.737051 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:52.736991 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:52.737359 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.737107 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:52.737571 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.737544 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:52.737641 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:52.737612 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:52.737696 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:52.737681 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:54.736929 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:54.736889 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:54.737429 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:54.736988 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:54.737429 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:54.737061 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:54.737429 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:54.737140 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:54.737429 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:54.737183 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:54.737429 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:54.737257 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:56.738220 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.736722 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:56.738220 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:56.737368 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:56.738220 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.737840 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:56.738220 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:56.737965 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:56.739336 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.738898 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:56.739336 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:56.738963 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:56.876715 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.876451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pthc2" event={"ID":"65e2fcb0-31cc-456c-a4e9-4896bfa4cc41","Type":"ContainerStarted","Data":"f63a1876dc7ad950668bd0407021fafd2eee67038a910fc6156b8acf3101485e"} Jul 08 16:34:56.878356 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.878326 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-22m2d" event={"ID":"c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc","Type":"ContainerStarted","Data":"fb7e134f56b50424a69ccc6e402cd87f77bf351411ddc8516e292ff22899ab9b"} Jul 08 16:34:56.880684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.880609 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lnq7s" event={"ID":"e10fb928-0f80-4c9e-a510-297440e94a5f","Type":"ContainerStarted","Data":"06bf3609b56a2ba08a9edecd321931332e7ffff2daaec72bf170b2fc745c50a7"} Jul 08 16:34:56.883178 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.883148 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" event={"ID":"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b","Type":"ContainerStarted","Data":"82388fbb60a4bebd5b63c13217d74f18272b04617cb8782b00ea501dac535ff8"} Jul 08 16:34:56.886165 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.886134 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61" containerID="295046b3fc5dadfe588e9ab370c66c415b9ae809c68cbb19eca19610b7ec13c4" exitCode=0 Jul 08 16:34:56.886364 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.886202 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" event={"ID":"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61","Type":"ContainerDied","Data":"295046b3fc5dadfe588e9ab370c66c415b9ae809c68cbb19eca19610b7ec13c4"} Jul 08 16:34:56.888593 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.888561 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pqjtx" event={"ID":"ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a","Type":"ContainerStarted","Data":"c46b6eae69a4019d837c027a8869cd23564ae514d92066490b0a4d51788d081e"} Jul 08 16:34:56.892097 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.891975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"c22500aea592d8ac0f8e965c69d6c1ba551c15afb46f8483ab5155ed7d3de823"} Jul 08 16:34:56.892464 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.892073 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"b5d260a5b5818efae198cf610c8a33a94847f6c9ab1dbd83463c8179c1231d8e"} Jul 08 16:34:56.892464 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.892265 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"1d3761c6deae2d402c3513aa4947c704dc519c5b98738e44e7c54b33605a0b64"} Jul 08 16:34:56.894152 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.894106 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s6xcp" event={"ID":"864a8a9f-7cae-43e4-9736-2174d574a7e8","Type":"ContainerStarted","Data":"afb82c501c5a1361ab5f08cf8d3d4a87011e1dad1470e5390295327ddd75c6f9"} Jul 08 16:34:56.916709 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.916613 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-254.ec2.internal" podStartSLOduration=20.916593858 podStartE2EDuration="20.916593858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:38.841114934 +0000 UTC m=+4.568039808" watchObservedRunningTime="2026-07-08 16:34:56.916593858 +0000 UTC m=+22.643518732" Jul 08 16:34:56.916982 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.916957 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pthc2" podStartSLOduration=3.805971219 podStartE2EDuration="22.916950845s" totalImagesPullingTime="19.110979626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:56.916485514 +0000 UTC m=+22.643410405" watchObservedRunningTime="2026-07-08 16:34:56.916950845 +0000 UTC m=+22.643875718" Jul 08 16:34:56.957750 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.957694 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s6xcp" podStartSLOduration=12.324338638 podStartE2EDuration="22.95767738s" totalImagesPullingTime="10.633338742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:56.95715569 +0000 UTC m=+22.684080564" watchObservedRunningTime="2026-07-08 16:34:56.95767738 +0000 UTC m=+22.684602261" Jul 08 16:34:56.977250 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.977195 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lnq7s" podStartSLOduration=2.820768354 podStartE2EDuration="21.977178406s" totalImagesPullingTime="19.156410052s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:56.976718176 +0000 UTC m=+22.703643048" watchObservedRunningTime="2026-07-08 16:34:56.977178406 +0000 UTC m=+22.704103278" Jul 08 16:34:56.997335 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:56.997259 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-22m2d" podStartSLOduration=3.891831616 podStartE2EDuration="22.997244087s" totalImagesPullingTime="19.105412471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:56.997094099 +0000 UTC m=+22.724018993" watchObservedRunningTime="2026-07-08 16:34:56.997244087 +0000 UTC m=+22.724168959" Jul 08 16:34:57.015346 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:57.015269 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pqjtx" podStartSLOduration=3.872194624 podStartE2EDuration="23.015254211s" totalImagesPullingTime="19.143059587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:57.014979515 +0000 UTC m=+22.741904388" watchObservedRunningTime="2026-07-08 16:34:57.015254211 +0000 UTC m=+22.742179084" Jul 08 16:34:57.897809 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:57.897771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gwjxc" event={"ID":"16660cb9-13e7-489c-bad0-08851d9f10b3","Type":"ContainerStarted","Data":"4009fc6d2ab5c4c2c9b18d2c372967ca51a5464c9b17392ff98ebe85d082a28b"} Jul 08 16:34:57.900660 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:57.900631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"2f47c4ed332b065e74bf9d9b845de4a3f1a91c612c16f777e145be4206180b61"} Jul 08 16:34:57.900792 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:57.900714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"f10a9f4f92acd85ed6125153085183a645ef6fc5af2eac95b95e885f1df323c1"} Jul 08 16:34:57.900792 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:57.900744 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"9c61e7a875a845ff1c0f64e2ca3282e1ce70cfe370f2875a9fdfb940e627edfb"} Jul 08 16:34:57.913916 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:57.913867 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gwjxc" podStartSLOduration=3.80830941 podStartE2EDuration="22.9138483s" totalImagesPullingTime="19.10553889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:57.913620142 +0000 UTC m=+23.640545035" watchObservedRunningTime="2026-07-08 16:34:57.9138483 +0000 UTC m=+23.640773172" Jul 08 16:34:58.736769 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:58.736724 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:34:58.736986 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:58.736726 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:34:58.736986 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:58.736843 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:34:58.736986 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:58.736878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:34:58.736986 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:58.736972 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:34:58.737125 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:34:58.737054 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:34:58.769670 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:58.769206 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 16:34:58.905210 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:58.905105 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" event={"ID":"1b906b0da08878c33265f8e1596ed7a5","Type":"ContainerStarted","Data":"688e0cc008d3b926cba45fddd8e984d20211c0bb084acfe54032522c22a80fbf"} Jul 08 16:34:58.908174 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:58.908141 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" event={"ID":"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b","Type":"ContainerStarted","Data":"9666d5f7cc3b56079506e800451f834feaa248d1fd9c7a49a430c9edd5572307"} Jul 08 16:34:58.921233 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:58.921154 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-254.ec2.internal" podStartSLOduration=22.921131683 podStartE2EDuration="22.921131683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:34:58.920795101 +0000 UTC m=+24.647719982" watchObservedRunningTime="2026-07-08 16:34:58.921131683 +0000 UTC m=+24.648056559" Jul 08 16:34:59.683610 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:59.682816 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T16:34:58.769225105Z","UUID":"a2f60d6d-8879-433a-baab-a6095ffbe254","Handler":null,"Name":"","Endpoint":""} Jul 08 16:34:59.684716 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:59.684694 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 16:34:59.684846 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:59.684739 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 16:34:59.914566 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:34:59.914524 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"584d59838ac772deed52d9fc9d2be688eeb72a797daaa9bd5bdfca8df459b780"} Jul 08 16:35:00.736437 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.736139 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:00.736671 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.736139 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:00.736671 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:00.736547 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:00.736671 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.736139 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:00.736671 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:00.736629 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:00.736862 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:00.736695 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:00.908369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.908333 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:35:00.909017 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.908994 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:35:00.920232 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.920200 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" event={"ID":"ae02f4f6-2137-47a4-98b0-cfc6f2f73f5b","Type":"ContainerStarted","Data":"46e5f5ccbcf81b5b3b1ca80aecb3f32a58f4c90bd52c28cb4f859c799a35c7ca"} Jul 08 16:35:00.920719 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.920379 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:35:00.920894 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.920872 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s6xcp" Jul 08 16:35:00.944932 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:00.944873 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chnr9" podStartSLOduration=4.467338132 podStartE2EDuration="26.944855923s" totalImagesPullingTime="22.477517791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:00.944668969 +0000 UTC m=+26.671593856" watchObservedRunningTime="2026-07-08 16:35:00.944855923 +0000 UTC m=+26.671780796" Jul 08 16:35:01.927720 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:01.927685 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" event={"ID":"823540fd-8449-4516-ab31-489f22b48084","Type":"ContainerStarted","Data":"c9d32abcc2beb8bc193f41a35a93c59c56ca3a9a369ac9ab110ee02cbb1a1b6c"} Jul 08 16:35:01.928671 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:01.928150 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:35:01.928671 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:01.928202 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:35:01.928671 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:01.928250 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:35:01.952010 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:01.951949 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" podStartSLOduration=7.768976926 podStartE2EDuration="26.951926695s" totalImagesPullingTime="19.182949769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:01.951500317 +0000 UTC m=+27.678425191" watchObservedRunningTime="2026-07-08 16:35:01.951926695 +0000 UTC m=+27.678851569" Jul 08 16:35:01.957550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:01.957523 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:35:01.957913 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:01.957759 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:35:02.736973 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:02.736933 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:02.737209 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:02.737048 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:02.737209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:02.736944 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:02.737209 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:02.737190 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:02.737399 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:02.737232 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:02.737399 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:02.737352 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:02.931669 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:02.931630 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61" containerID="b1e74b66abac5e6513ccbd8cde0be02781e2c23351de2a670937225f11e46d6a" exitCode=0 Jul 08 16:35:02.932092 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:02.931724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" event={"ID":"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61","Type":"ContainerDied","Data":"b1e74b66abac5e6513ccbd8cde0be02781e2c23351de2a670937225f11e46d6a"} Jul 08 16:35:03.936711 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:03.936671 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61" containerID="d8bd1560874ae9391ec68141582ed19e292dcb43935a378df7c5632ce852523a" exitCode=0 Jul 08 16:35:03.937161 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:03.936759 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" event={"ID":"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61","Type":"ContainerDied","Data":"d8bd1560874ae9391ec68141582ed19e292dcb43935a378df7c5632ce852523a"} Jul 08 16:35:03.942952 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:03.942932 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pqjtx_ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a/dns-node-resolver/0.log" Jul 08 16:35:04.086538 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:04.086442 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-knbpm"] Jul 08 16:35:04.086737 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:04.086575 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:04.086737 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:04.086686 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:04.087144 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:04.087119 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9gh26"] Jul 08 16:35:04.087263 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:04.087212 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:04.087361 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:04.087315 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:04.102375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:04.102343 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zbrcz"] Jul 08 16:35:04.102533 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:04.102449 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:04.102575 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:04.102536 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:04.716557 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:04.716528 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-22m2d_c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc/node-ca/0.log" Jul 08 16:35:05.736579 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:05.736383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:05.737027 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:05.736383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:05.737027 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:05.736660 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:05.737027 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:05.736383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:05.737027 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:05.736764 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:05.737027 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:05.736814 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:05.942925 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:05.942893 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61" containerID="ada0741d287560200d53a2a0926df944d0f43ea11610e50e695ed1712e8cb1b0" exitCode=0 Jul 08 16:35:05.943104 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:05.942940 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" event={"ID":"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61","Type":"ContainerDied","Data":"ada0741d287560200d53a2a0926df944d0f43ea11610e50e695ed1712e8cb1b0"} Jul 08 16:35:07.736209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:07.736169 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:07.737022 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:07.736304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:07.737022 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:07.736321 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:07.737022 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:07.736336 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:07.737022 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:07.736411 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:07.737022 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:07.736490 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:08.470409 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:08.470369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:08.470595 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:08.470423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:08.470595 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:08.470530 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:35:08.470595 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:08.470535 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:35:08.470595 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:08.470588 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret podName:1cdda881-c366-4e95-ab7e-6491a2e4c00b nodeName:}" failed. No retries permitted until 2026-07-08 16:35:40.470574658 +0000 UTC m=+66.197499509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret") pod "global-pull-secret-syncer-zbrcz" (UID: "1cdda881-c366-4e95-ab7e-6491a2e4c00b") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:35:08.470727 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:08.470604 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs podName:7109969a-f717-4285-b47f-261f6b3353a2 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:40.470596361 +0000 UTC m=+66.197521211 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs") pod "network-metrics-daemon-9gh26" (UID: "7109969a-f717-4285-b47f-261f6b3353a2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:35:08.571496 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:08.571452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:08.571666 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:08.571629 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:35:08.571666 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:08.571652 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:35:08.571666 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:08.571662 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vnjbc for pod openshift-network-diagnostics/network-check-target-knbpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:35:08.571786 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:08.571726 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc podName:2812fd9f-259d-44ae-91fa-d839fe4b94b9 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:40.571707265 +0000 UTC m=+66.298632118 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vnjbc" (UniqueName: "kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc") pod "network-check-target-knbpm" (UID: "2812fd9f-259d-44ae-91fa-d839fe4b94b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:35:09.736575 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:09.736540 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:09.737117 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:09.736588 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:09.737117 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:09.736588 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:09.737117 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:09.736670 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:09.737117 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:09.736793 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:09.737117 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:09.736912 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:10.958839 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:10.958801 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61" containerID="f3075a79cf3ac6e73c60b2098474faf1bea4a98a3dfe69bca159a099213d7bda" exitCode=0 Jul 08 16:35:10.959248 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:10.958860 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" event={"ID":"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61","Type":"ContainerDied","Data":"f3075a79cf3ac6e73c60b2098474faf1bea4a98a3dfe69bca159a099213d7bda"} Jul 08 16:35:11.736449 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:11.736407 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:11.736643 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:11.736407 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:11.736643 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:11.736526 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:11.736643 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:11.736598 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:11.736643 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:11.736407 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:11.736782 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:11.736677 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:11.965558 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:11.965526 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61" containerID="096373f421fab1ba54eb8a1f477d4c52408d1b0343d6043a2f1c6ce3477cfe87" exitCode=0 Jul 08 16:35:11.965918 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:11.965590 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" event={"ID":"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61","Type":"ContainerDied","Data":"096373f421fab1ba54eb8a1f477d4c52408d1b0343d6043a2f1c6ce3477cfe87"} Jul 08 16:35:12.971777 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:12.971560 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" event={"ID":"0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61","Type":"ContainerStarted","Data":"1eaddf1a49efde4791a565ed57a7c2d5736fed0248d9640b4d00d910a884cc65"} Jul 08 16:35:13.003225 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:13.003162 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5bnw6" podStartSLOduration=8.541963875 podStartE2EDuration="39.003138806s" totalImagesPullingTime="30.461174931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:13.002684899 +0000 UTC m=+38.729609771" watchObservedRunningTime="2026-07-08 16:35:13.003138806 +0000 UTC m=+38.730063680" Jul 08 16:35:13.736609 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:13.736531 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:13.736762 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:13.736531 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:13.736762 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:13.736640 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:13.736851 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:13.736544 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:13.736851 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:13.736776 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:13.736851 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:13.736845 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:15.736162 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:15.736117 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:15.736162 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:15.736148 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:15.736614 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:15.736234 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:15.736614 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:15.736247 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:15.736614 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:15.736315 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:15.736614 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:15.736405 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:17.736130 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:17.736094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:17.736130 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:17.736116 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:17.736647 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:17.736167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:17.736647 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:17.736259 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:17.736647 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:17.736313 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:17.736647 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:17.736385 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:19.736449 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:19.736404 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:19.736882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:19.736404 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:19.736882 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:19.736531 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:19.736882 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:19.736585 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:19.736882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:19.736404 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:19.736882 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:19.736691 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:21.736047 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:21.736001 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:21.736047 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:21.736046 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:21.736685 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:21.736052 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:21.736685 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:21.736133 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:21.736685 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:21.736202 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:21.736685 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:21.736297 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:23.736366 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:23.736325 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:23.736851 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:23.736333 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:23.736851 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:23.736443 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:23.736851 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:23.736437 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:23.736851 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:23.736526 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:23.736851 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:23.736595 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:25.736012 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:25.735970 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:25.736460 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:25.736079 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:25.736460 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:25.736093 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:25.736460 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:25.735980 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:25.736460 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:25.736181 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:25.736460 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:25.736224 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:27.736641 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:27.736606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:27.736641 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:27.736639 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:27.737129 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:27.736712 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:27.737129 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:27.736734 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbrcz" podUID="1cdda881-c366-4e95-ab7e-6491a2e4c00b" Jul 08 16:35:27.737129 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:27.736784 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knbpm" podUID="2812fd9f-259d-44ae-91fa-d839fe4b94b9" Jul 08 16:35:27.737129 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:27.736864 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gh26" podUID="7109969a-f717-4285-b47f-261f6b3353a2" Jul 08 16:35:28.054925 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.054828 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-254.ec2.internal" event="NodeReady" Jul 08 16:35:28.055065 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.054956 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 16:35:28.170567 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.170531 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vqswf"] Jul 08 16:35:28.195477 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.195441 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gxg57"] Jul 08 16:35:28.195635 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.195587 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.198297 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.198250 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 16:35:28.198424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.198308 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jflkn\"" Jul 08 16:35:28.198424 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.198308 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 16:35:28.215585 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.215510 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vqswf"] Jul 08 16:35:28.215738 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.215603 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxg57"] Jul 08 16:35:28.215738 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.215645 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxg57" Jul 08 16:35:28.217712 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.217684 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 16:35:28.217712 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.217703 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 16:35:28.217965 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.217913 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lgg5h\"" Jul 08 16:35:28.218028 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.217965 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 16:35:28.271211 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.271175 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-726rr"] Jul 08 16:35:28.289804 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.289776 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-726rr"] Jul 08 16:35:28.289957 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.289889 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.291810 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.291779 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 16:35:28.291985 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.291912 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 16:35:28.292240 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.292223 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 16:35:28.292308 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.292267 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 16:35:28.292429 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.292404 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5r9vg\"" Jul 08 16:35:28.312431 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.312342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-metrics-tls\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.312431 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.312411 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30873582-3370-4d20-a0e9-f9bb1dd67dcd-cert\") pod \"ingress-canary-gxg57\" (UID: \"30873582-3370-4d20-a0e9-f9bb1dd67dcd\") " pod="openshift-ingress-canary/ingress-canary-gxg57" Jul 08 16:35:28.312638 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.312450 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-config-volume\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.312638 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.312471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-tmp-dir\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.312638 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.312525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g87n\" (UniqueName: \"kubernetes.io/projected/30873582-3370-4d20-a0e9-f9bb1dd67dcd-kube-api-access-8g87n\") pod \"ingress-canary-gxg57\" (UID: \"30873582-3370-4d20-a0e9-f9bb1dd67dcd\") " pod="openshift-ingress-canary/ingress-canary-gxg57" Jul 08 16:35:28.312638 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.312561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6tdw\" (UniqueName: \"kubernetes.io/projected/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-kube-api-access-m6tdw\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.413317 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghhq2\" (UniqueName: \"kubernetes.io/projected/97471e01-6997-4859-9309-8e8c09490e73-kube-api-access-ghhq2\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.413317 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413320 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6tdw\" (UniqueName: \"kubernetes.io/projected/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-kube-api-access-m6tdw\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.413576 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413351 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-metrics-tls\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.413576 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413370 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/97471e01-6997-4859-9309-8e8c09490e73-crio-socket\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.413576 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97471e01-6997-4859-9309-8e8c09490e73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.413576 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97471e01-6997-4859-9309-8e8c09490e73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.413576 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30873582-3370-4d20-a0e9-f9bb1dd67dcd-cert\") pod \"ingress-canary-gxg57\" (UID: \"30873582-3370-4d20-a0e9-f9bb1dd67dcd\") " pod="openshift-ingress-canary/ingress-canary-gxg57" Jul 08 16:35:28.413576 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-config-volume\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.413863 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413590 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-tmp-dir\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.413863 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g87n\" (UniqueName: \"kubernetes.io/projected/30873582-3370-4d20-a0e9-f9bb1dd67dcd-kube-api-access-8g87n\") pod \"ingress-canary-gxg57\" (UID: \"30873582-3370-4d20-a0e9-f9bb1dd67dcd\") " pod="openshift-ingress-canary/ingress-canary-gxg57" Jul 08 16:35:28.413863 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/97471e01-6997-4859-9309-8e8c09490e73-data-volume\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.413985 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.413902 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-tmp-dir\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.414126 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.414104 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-config-volume\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.417507 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.417486 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-metrics-tls\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.417694 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.417674 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30873582-3370-4d20-a0e9-f9bb1dd67dcd-cert\") pod \"ingress-canary-gxg57\" (UID: \"30873582-3370-4d20-a0e9-f9bb1dd67dcd\") " pod="openshift-ingress-canary/ingress-canary-gxg57" Jul 08 16:35:28.422011 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.421977 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6tdw\" (UniqueName: \"kubernetes.io/projected/c51f0313-ba74-44f2-9ea8-068e01c1cf3c-kube-api-access-m6tdw\") pod \"dns-default-vqswf\" (UID: \"c51f0313-ba74-44f2-9ea8-068e01c1cf3c\") " pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.422391 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.422373 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g87n\" (UniqueName: \"kubernetes.io/projected/30873582-3370-4d20-a0e9-f9bb1dd67dcd-kube-api-access-8g87n\") pod \"ingress-canary-gxg57\" (UID: \"30873582-3370-4d20-a0e9-f9bb1dd67dcd\") " pod="openshift-ingress-canary/ingress-canary-gxg57" Jul 08 16:35:28.505412 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.505370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:28.514613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.514579 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghhq2\" (UniqueName: \"kubernetes.io/projected/97471e01-6997-4859-9309-8e8c09490e73-kube-api-access-ghhq2\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.514750 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.514626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/97471e01-6997-4859-9309-8e8c09490e73-crio-socket\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.514802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.514746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97471e01-6997-4859-9309-8e8c09490e73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.514802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.514776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97471e01-6997-4859-9309-8e8c09490e73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.514890 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.514835 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/97471e01-6997-4859-9309-8e8c09490e73-crio-socket\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.514890 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.514855 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/97471e01-6997-4859-9309-8e8c09490e73-data-volume\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.515151 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.515131 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/97471e01-6997-4859-9309-8e8c09490e73-data-volume\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.515295 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.515240 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97471e01-6997-4859-9309-8e8c09490e73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.517214 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.517197 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97471e01-6997-4859-9309-8e8c09490e73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.522358 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.522334 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghhq2\" (UniqueName: \"kubernetes.io/projected/97471e01-6997-4859-9309-8e8c09490e73-kube-api-access-ghhq2\") pod \"insights-runtime-extractor-726rr\" (UID: \"97471e01-6997-4859-9309-8e8c09490e73\") " pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.524302 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.524261 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxg57" Jul 08 16:35:28.599020 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.598985 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-726rr" Jul 08 16:35:28.717168 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.716874 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vqswf"] Jul 08 16:35:28.722651 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:28.722605 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51f0313_ba74_44f2_9ea8_068e01c1cf3c.slice/crio-eedc871e2dbe6822d7186d1332c77dd75ef1945fe70404f0cd00382a4e6547de WatchSource:0}: Error finding container eedc871e2dbe6822d7186d1332c77dd75ef1945fe70404f0cd00382a4e6547de: Status 404 returned error can't find the container with id eedc871e2dbe6822d7186d1332c77dd75ef1945fe70404f0cd00382a4e6547de Jul 08 16:35:28.729462 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.729430 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxg57"] Jul 08 16:35:28.733294 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:28.733250 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30873582_3370_4d20_a0e9_f9bb1dd67dcd.slice/crio-562816a9d9391dd4672aa3eca36cefdb5c6254a81640e83d2e5bb01f82fcdd61 WatchSource:0}: Error finding container 562816a9d9391dd4672aa3eca36cefdb5c6254a81640e83d2e5bb01f82fcdd61: Status 404 returned error can't find the container with id 562816a9d9391dd4672aa3eca36cefdb5c6254a81640e83d2e5bb01f82fcdd61 Jul 08 16:35:28.771669 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:28.771640 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-726rr"] Jul 08 16:35:28.774819 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:28.774791 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97471e01_6997_4859_9309_8e8c09490e73.slice/crio-1230a9026ea3e9f9fcae726c0cad0b4f61fd33db050e36882e7a89f6223e79c9 WatchSource:0}: Error finding container 1230a9026ea3e9f9fcae726c0cad0b4f61fd33db050e36882e7a89f6223e79c9: Status 404 returned error can't find the container with id 1230a9026ea3e9f9fcae726c0cad0b4f61fd33db050e36882e7a89f6223e79c9 Jul 08 16:35:29.008839 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.008798 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-726rr" event={"ID":"97471e01-6997-4859-9309-8e8c09490e73","Type":"ContainerStarted","Data":"1230a9026ea3e9f9fcae726c0cad0b4f61fd33db050e36882e7a89f6223e79c9"} Jul 08 16:35:29.009955 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.009928 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxg57" event={"ID":"30873582-3370-4d20-a0e9-f9bb1dd67dcd","Type":"ContainerStarted","Data":"562816a9d9391dd4672aa3eca36cefdb5c6254a81640e83d2e5bb01f82fcdd61"} Jul 08 16:35:29.010963 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.010941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqswf" event={"ID":"c51f0313-ba74-44f2-9ea8-068e01c1cf3c","Type":"ContainerStarted","Data":"eedc871e2dbe6822d7186d1332c77dd75ef1945fe70404f0cd00382a4e6547de"} Jul 08 16:35:29.736595 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.736555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:29.737053 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.737036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:29.737507 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.737473 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:29.741389 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.741365 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 16:35:29.741666 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.741650 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 16:35:29.742164 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.742144 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 16:35:29.742362 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.742342 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 16:35:29.742431 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.742367 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-krkqn\"" Jul 08 16:35:29.742592 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:29.742575 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j7mv4\"" Jul 08 16:35:33.949125 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:33.949098 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xcqgq" Jul 08 16:35:35.030200 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.030149 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-726rr" event={"ID":"97471e01-6997-4859-9309-8e8c09490e73","Type":"ContainerStarted","Data":"ecc99b238b413bfc04c0eea945180810cea49b336ba7a9a855e7c5bdcd175b4d"} Jul 08 16:35:35.031898 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.031867 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxg57" event={"ID":"30873582-3370-4d20-a0e9-f9bb1dd67dcd","Type":"ContainerStarted","Data":"e84cd62f520a08452a4bc84002500679ad268cf8c022e844c7276b2ff91d1e23"} Jul 08 16:35:35.033675 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.033648 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqswf" event={"ID":"c51f0313-ba74-44f2-9ea8-068e01c1cf3c","Type":"ContainerStarted","Data":"0e25e4a65c68b9ba08946d59522ec872c9bd662a67fcfa87664c55cdb58eee99"} Jul 08 16:35:35.033792 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.033695 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqswf" event={"ID":"c51f0313-ba74-44f2-9ea8-068e01c1cf3c","Type":"ContainerStarted","Data":"97f5a9722c5bc8f840c5eda0f7eca8de3733ac96d34638ed2fc689efaf5e3f4d"} Jul 08 16:35:35.033792 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.033727 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:35.047903 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.047636 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gxg57" podStartSLOduration=1.389173241 podStartE2EDuration="7.047614927s" totalImagesPullingTime="5.658441686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:35.046941892 +0000 UTC m=+60.773866782" watchObservedRunningTime="2026-07-08 16:35:35.047614927 +0000 UTC m=+60.774539801" Jul 08 16:35:35.062817 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.062763 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vqswf" podStartSLOduration=1.393327017 podStartE2EDuration="7.06274151s" totalImagesPullingTime="5.669414493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:35.061735741 +0000 UTC m=+60.788660635" watchObservedRunningTime="2026-07-08 16:35:35.06274151 +0000 UTC m=+60.789666360" Jul 08 16:35:35.990685 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.990646 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7"] Jul 08 16:35:35.994436 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.994408 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" Jul 08 16:35:35.996574 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.996548 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 16:35:35.996731 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:35.996664 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-m9cwv\"" Jul 08 16:35:36.001568 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.001546 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7"] Jul 08 16:35:36.038336 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.038303 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-726rr" event={"ID":"97471e01-6997-4859-9309-8e8c09490e73","Type":"ContainerStarted","Data":"bcb04d3517d5fd2a3c38815ab7ad33b6d6da666db866668a278332cf8cf440af"} Jul 08 16:35:36.074578 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.074538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/10450850-4160-4380-b409-6ed4722dfd45-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-tkqm7\" (UID: \"10450850-4160-4380-b409-6ed4722dfd45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" Jul 08 16:35:36.175393 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.175351 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/10450850-4160-4380-b409-6ed4722dfd45-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-tkqm7\" (UID: \"10450850-4160-4380-b409-6ed4722dfd45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" Jul 08 16:35:36.178426 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.178384 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/10450850-4160-4380-b409-6ed4722dfd45-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-tkqm7\" (UID: \"10450850-4160-4380-b409-6ed4722dfd45\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" Jul 08 16:35:36.304119 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.304032 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" Jul 08 16:35:36.351458 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.351422 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d89d49546-m8t4f"] Jul 08 16:35:36.354759 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.354731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.363945 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.362611 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 16:35:36.363945 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.362961 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 16:35:36.363945 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.363192 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 16:35:36.364763 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.364090 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 16:35:36.364763 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.364408 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 16:35:36.364763 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.364595 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 16:35:36.364763 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.364722 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2bkm5\"" Jul 08 16:35:36.364986 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.364879 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 16:35:36.376079 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.376031 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d89d49546-m8t4f"] Jul 08 16:35:36.457349 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.457317 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7"] Jul 08 16:35:36.460834 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:36.460807 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10450850_4160_4380_b409_6ed4722dfd45.slice/crio-350e1cec38ce650f89ac81f5c5d2835ee320f6418e7b3dfb9ef2904f0421fd4f WatchSource:0}: Error finding container 350e1cec38ce650f89ac81f5c5d2835ee320f6418e7b3dfb9ef2904f0421fd4f: Status 404 returned error can't find the container with id 350e1cec38ce650f89ac81f5c5d2835ee320f6418e7b3dfb9ef2904f0421fd4f Jul 08 16:35:36.477609 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.477576 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-serving-cert\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.477609 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.477609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96shx\" (UniqueName: \"kubernetes.io/projected/f9c47575-c427-42f8-92d3-4f7986f4e618-kube-api-access-96shx\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.477819 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.477670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-oauth-serving-cert\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.477819 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.477719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-service-ca\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.477819 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.477748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-oauth-config\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.477819 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.477767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-console-config\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.578200 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.578110 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-oauth-config\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.578200 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.578159 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-console-config\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.578449 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.578207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-serving-cert\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.578449 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.578329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96shx\" (UniqueName: \"kubernetes.io/projected/f9c47575-c427-42f8-92d3-4f7986f4e618-kube-api-access-96shx\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.578449 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.578413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-oauth-serving-cert\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.578607 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.578486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-service-ca\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.579076 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.578988 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-console-config\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.579076 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.579062 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-service-ca\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.579248 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.579149 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-oauth-serving-cert\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.581160 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.581132 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-oauth-config\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.581260 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.581221 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-serving-cert\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.588495 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.588470 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96shx\" (UniqueName: \"kubernetes.io/projected/f9c47575-c427-42f8-92d3-4f7986f4e618-kube-api-access-96shx\") pod \"console-7d89d49546-m8t4f\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.669111 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.669071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:36.836449 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:36.836331 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d89d49546-m8t4f"] Jul 08 16:35:36.841783 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:36.841526 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c47575_c427_42f8_92d3_4f7986f4e618.slice/crio-2449f81ea5158cef29aab4ecd3335d2c1d28015137915a69c62f835396445137 WatchSource:0}: Error finding container 2449f81ea5158cef29aab4ecd3335d2c1d28015137915a69c62f835396445137: Status 404 returned error can't find the container with id 2449f81ea5158cef29aab4ecd3335d2c1d28015137915a69c62f835396445137 Jul 08 16:35:37.041985 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:37.041947 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d89d49546-m8t4f" event={"ID":"f9c47575-c427-42f8-92d3-4f7986f4e618","Type":"ContainerStarted","Data":"2449f81ea5158cef29aab4ecd3335d2c1d28015137915a69c62f835396445137"} Jul 08 16:35:37.043200 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:37.043173 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" event={"ID":"10450850-4160-4380-b409-6ed4722dfd45","Type":"ContainerStarted","Data":"350e1cec38ce650f89ac81f5c5d2835ee320f6418e7b3dfb9ef2904f0421fd4f"} Jul 08 16:35:39.055940 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:39.055904 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-726rr" event={"ID":"97471e01-6997-4859-9309-8e8c09490e73","Type":"ContainerStarted","Data":"a48f3b22708eecde8830f0ae6ca41265bd442af902b3de1e57d8d4ff79c36fad"} Jul 08 16:35:39.074416 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:39.074340 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-726rr" podStartSLOduration=1.839917477 podStartE2EDuration="11.074314398s" totalImagesPullingTime="9.234396921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:35:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:39.073498532 +0000 UTC m=+64.800423406" watchObservedRunningTime="2026-07-08 16:35:39.074314398 +0000 UTC m=+64.801239269" Jul 08 16:35:40.061574 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.061536 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" event={"ID":"10450850-4160-4380-b409-6ed4722dfd45","Type":"ContainerStarted","Data":"e34ac29d1e34ee23a8d1677ffb7ca574bb40634cfc2f4d531f15c8724d2cfcc2"} Jul 08 16:35:40.062005 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.061921 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" Jul 08 16:35:40.067238 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.067215 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" Jul 08 16:35:40.078864 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.078795 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-tkqm7" podStartSLOduration=2.537517053 podStartE2EDuration="5.078771991s" totalImagesPullingTime="2.541254938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:40.078045069 +0000 UTC m=+65.804969942" watchObservedRunningTime="2026-07-08 16:35:40.078771991 +0000 UTC m=+65.805696866" Jul 08 16:35:40.513334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.513266 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:40.513334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.513341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:40.516517 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.516491 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 16:35:40.517519 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.517497 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 16:35:40.526966 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.526935 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cdda881-c366-4e95-ab7e-6491a2e4c00b-original-pull-secret\") pod \"global-pull-secret-syncer-zbrcz\" (UID: \"1cdda881-c366-4e95-ab7e-6491a2e4c00b\") " pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:40.526966 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.526956 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7109969a-f717-4285-b47f-261f6b3353a2-metrics-certs\") pod \"network-metrics-daemon-9gh26\" (UID: \"7109969a-f717-4285-b47f-261f6b3353a2\") " pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:40.557475 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.557438 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-krkqn\"" Jul 08 16:35:40.565196 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.565162 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gh26" Jul 08 16:35:40.583860 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.583829 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbrcz" Jul 08 16:35:40.614475 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.614435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:40.616810 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.616779 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 16:35:40.627849 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.627821 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 16:35:40.639429 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.639392 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjbc\" (UniqueName: \"kubernetes.io/projected/2812fd9f-259d-44ae-91fa-d839fe4b94b9-kube-api-access-vnjbc\") pod \"network-check-target-knbpm\" (UID: \"2812fd9f-259d-44ae-91fa-d839fe4b94b9\") " pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:40.870037 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.869986 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zbrcz"] Jul 08 16:35:40.872615 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:40.872585 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cdda881_c366_4e95_ab7e_6491a2e4c00b.slice/crio-1dbd4fdfa74c4c608f6d83ab2d741651d0ae1574e8aaa9479bc0b1cbe761789b WatchSource:0}: Error finding container 1dbd4fdfa74c4c608f6d83ab2d741651d0ae1574e8aaa9479bc0b1cbe761789b: Status 404 returned error can't find the container with id 1dbd4fdfa74c4c608f6d83ab2d741651d0ae1574e8aaa9479bc0b1cbe761789b Jul 08 16:35:40.877846 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.877816 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j7mv4\"" Jul 08 16:35:40.884114 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.884082 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9gh26"] Jul 08 16:35:40.886438 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:40.886419 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:40.887463 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:40.887439 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7109969a_f717_4285_b47f_261f6b3353a2.slice/crio-6528c0b3040b17fc6c3381cb88bd9a83e697c040943fa09285b875824b157e8f WatchSource:0}: Error finding container 6528c0b3040b17fc6c3381cb88bd9a83e697c040943fa09285b875824b157e8f: Status 404 returned error can't find the container with id 6528c0b3040b17fc6c3381cb88bd9a83e697c040943fa09285b875824b157e8f Jul 08 16:35:41.016173 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.016136 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-knbpm"] Jul 08 16:35:41.019890 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:41.019861 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2812fd9f_259d_44ae_91fa_d839fe4b94b9.slice/crio-befdde0ad3086e614810cd5ede5770ceb055651c581a24aa2d88a8e39e68408d WatchSource:0}: Error finding container befdde0ad3086e614810cd5ede5770ceb055651c581a24aa2d88a8e39e68408d: Status 404 returned error can't find the container with id befdde0ad3086e614810cd5ede5770ceb055651c581a24aa2d88a8e39e68408d Jul 08 16:35:41.059192 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.059107 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-896h4"] Jul 08 16:35:41.063519 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.063497 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.065467 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.065417 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 16:35:41.065761 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.065735 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 16:35:41.065873 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.065860 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 16:35:41.065958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.065948 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-4lmts\"" Jul 08 16:35:41.066016 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.065864 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 16:35:41.066263 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.066243 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 16:35:41.067205 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.067184 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d89d49546-m8t4f" event={"ID":"f9c47575-c427-42f8-92d3-4f7986f4e618","Type":"ContainerStarted","Data":"a0401fd43077a7f70d4892388a9909caf4eec82b9ba9b7d679bb4dcee9134981"} Jul 08 16:35:41.068444 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.068419 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-knbpm" event={"ID":"2812fd9f-259d-44ae-91fa-d839fe4b94b9","Type":"ContainerStarted","Data":"befdde0ad3086e614810cd5ede5770ceb055651c581a24aa2d88a8e39e68408d"} Jul 08 16:35:41.069657 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.069635 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zbrcz" event={"ID":"1cdda881-c366-4e95-ab7e-6491a2e4c00b","Type":"ContainerStarted","Data":"1dbd4fdfa74c4c608f6d83ab2d741651d0ae1574e8aaa9479bc0b1cbe761789b"} Jul 08 16:35:41.071316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.071253 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gh26" event={"ID":"7109969a-f717-4285-b47f-261f6b3353a2","Type":"ContainerStarted","Data":"6528c0b3040b17fc6c3381cb88bd9a83e697c040943fa09285b875824b157e8f"} Jul 08 16:35:41.071760 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.071741 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-896h4"] Jul 08 16:35:41.098615 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.098566 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d89d49546-m8t4f" podStartSLOduration=1.156647492 podStartE2EDuration="5.098547971s" totalImagesPullingTime="3.941900479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:41.097329758 +0000 UTC m=+66.824254651" watchObservedRunningTime="2026-07-08 16:35:41.098547971 +0000 UTC m=+66.825472890" Jul 08 16:35:41.219434 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.219392 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/46e0a83b-036d-43d8-b6d2-ac54137630cb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.219642 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.219459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgn7j\" (UniqueName: \"kubernetes.io/projected/46e0a83b-036d-43d8-b6d2-ac54137630cb-kube-api-access-hgn7j\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.219642 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.219528 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e0a83b-036d-43d8-b6d2-ac54137630cb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.219642 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.219586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e0a83b-036d-43d8-b6d2-ac54137630cb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.320845 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.320740 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e0a83b-036d-43d8-b6d2-ac54137630cb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.320845 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.320822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/46e0a83b-036d-43d8-b6d2-ac54137630cb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.321083 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.320856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgn7j\" (UniqueName: \"kubernetes.io/projected/46e0a83b-036d-43d8-b6d2-ac54137630cb-kube-api-access-hgn7j\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.321083 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.320886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e0a83b-036d-43d8-b6d2-ac54137630cb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.321855 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.321817 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e0a83b-036d-43d8-b6d2-ac54137630cb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.323680 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.323652 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e0a83b-036d-43d8-b6d2-ac54137630cb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.323873 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.323852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/46e0a83b-036d-43d8-b6d2-ac54137630cb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.328976 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.328951 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgn7j\" (UniqueName: \"kubernetes.io/projected/46e0a83b-036d-43d8-b6d2-ac54137630cb-kube-api-access-hgn7j\") pod \"prometheus-operator-567f7b8b7c-896h4\" (UID: \"46e0a83b-036d-43d8-b6d2-ac54137630cb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.374904 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.374861 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" Jul 08 16:35:41.538514 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:41.538135 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-896h4"] Jul 08 16:35:42.081491 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:42.081447 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" event={"ID":"46e0a83b-036d-43d8-b6d2-ac54137630cb","Type":"ContainerStarted","Data":"b51e22e6ed551923ca23b34421cab81cff7cb904028a08cfd91c1134c212b561"} Jul 08 16:35:43.091326 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:43.090345 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gh26" event={"ID":"7109969a-f717-4285-b47f-261f6b3353a2","Type":"ContainerStarted","Data":"68cb5f278e23f1f51fbfc9ccae1b82a5cad29eac4feb67847d3f5534ef74ac29"} Jul 08 16:35:43.091326 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:43.090420 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gh26" event={"ID":"7109969a-f717-4285-b47f-261f6b3353a2","Type":"ContainerStarted","Data":"3610d884c348b3022d8aade9904feab08b75c8aa08b45e7fcc805a0ff37d1a10"} Jul 08 16:35:43.110688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:43.110609 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9gh26" podStartSLOduration=66.879966728 podStartE2EDuration="1m8.110591683s" totalImagesPullingTime="1.230624955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:43.108859396 +0000 UTC m=+68.835784282" watchObservedRunningTime="2026-07-08 16:35:43.110591683 +0000 UTC m=+68.837516555" Jul 08 16:35:44.096802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:44.096754 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" event={"ID":"46e0a83b-036d-43d8-b6d2-ac54137630cb","Type":"ContainerStarted","Data":"b1caa487797c016c22c2fa1c6e3741e6ec35e8fe8fd11a4760bfc5827881836b"} Jul 08 16:35:44.097255 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:44.096826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" event={"ID":"46e0a83b-036d-43d8-b6d2-ac54137630cb","Type":"ContainerStarted","Data":"327f53862e67aa33bfcb930b6114de42b9d966a0b814228a9c0bc66996732ed5"} Jul 08 16:35:44.116195 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:44.116142 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-896h4" podStartSLOduration=1.442165543 podStartE2EDuration="3.116120836s" totalImagesPullingTime="1.673955293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:44.114769107 +0000 UTC m=+69.841693981" watchObservedRunningTime="2026-07-08 16:35:44.116120836 +0000 UTC m=+69.843045708" Jul 08 16:35:45.041629 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:45.041588 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vqswf" Jul 08 16:35:46.106369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.106325 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-knbpm" event={"ID":"2812fd9f-259d-44ae-91fa-d839fe4b94b9","Type":"ContainerStarted","Data":"f1e6d18756c4707b5b2f2d73043bce9138f17e8df9e430181568f1de7b43e582"} Jul 08 16:35:46.106844 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.106490 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:35:46.107846 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.107821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zbrcz" event={"ID":"1cdda881-c366-4e95-ab7e-6491a2e4c00b","Type":"ContainerStarted","Data":"6be59e72f7de13646700014dfbcaf9173e6d6fcb7a0db30815223128c5a0b1f0"} Jul 08 16:35:46.129060 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.129006 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-knbpm" podStartSLOduration=66.23832077 podStartE2EDuration="1m11.12899166s" totalImagesPullingTime="4.89067089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:46.128655397 +0000 UTC m=+71.855580271" watchObservedRunningTime="2026-07-08 16:35:46.12899166 +0000 UTC m=+71.855916532" Jul 08 16:35:46.151636 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.151570 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zbrcz" podStartSLOduration=66.109682914 podStartE2EDuration="1m11.151546392s" totalImagesPullingTime="5.041863478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:46.15041199 +0000 UTC m=+71.877336864" watchObservedRunningTime="2026-07-08 16:35:46.151546392 +0000 UTC m=+71.878471267" Jul 08 16:35:46.484562 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.484521 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8"] Jul 08 16:35:46.487869 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.487842 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.492823 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.492784 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h"] Jul 08 16:35:46.493412 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.493392 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-m4zkv\"" Jul 08 16:35:46.493881 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.493866 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 16:35:46.494301 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.494259 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 16:35:46.495211 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.495192 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 16:35:46.496351 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.496335 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.499698 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.499669 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 16:35:46.499806 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.499669 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 16:35:46.499806 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.499788 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-m5zjz\"" Jul 08 16:35:46.520856 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.520818 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h"] Jul 08 16:35:46.521903 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.521878 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8"] Jul 08 16:35:46.537790 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.537757 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gm8gv"] Jul 08 16:35:46.540934 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.540910 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.544438 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.544257 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 16:35:46.544438 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.544378 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 16:35:46.544627 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.544517 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 16:35:46.544627 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.544270 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8djnp\"" Jul 08 16:35:46.567641 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567605 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-tls\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.567641 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567645 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-sys\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.567838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67cb91da-218f-41a7-bcba-cd2eb171d744-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.567838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.567838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.567838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.567838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567781 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-wtmp\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.567838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szf6x\" (UniqueName: \"kubernetes.io/projected/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-api-access-szf6x\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.567838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67cb91da-218f-41a7-bcba-cd2eb171d744-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-metrics-client-ca\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42gh8\" (UniqueName: \"kubernetes.io/projected/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-kube-api-access-42gh8\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.567985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cb91da-218f-41a7-bcba-cd2eb171d744-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.568012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrp92\" (UniqueName: \"kubernetes.io/projected/67cb91da-218f-41a7-bcba-cd2eb171d744-kube-api-access-xrp92\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.568041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.568057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-root\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.568095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.568077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-textfile\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669146 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.669146 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669179 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-wtmp\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szf6x\" (UniqueName: \"kubernetes.io/projected/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-api-access-szf6x\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669256 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67cb91da-218f-41a7-bcba-cd2eb171d744-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-metrics-client-ca\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669406 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-wtmp\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669453 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42gh8\" (UniqueName: \"kubernetes.io/projected/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-kube-api-access-42gh8\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cb91da-218f-41a7-bcba-cd2eb171d744-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrp92\" (UniqueName: \"kubernetes.io/projected/67cb91da-218f-41a7-bcba-cd2eb171d744-kube-api-access-xrp92\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669553 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-root\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-textfile\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669631 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669653 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-tls\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-sys\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67cb91da-218f-41a7-bcba-cd2eb171d744-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:46.669836 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669875 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:46.669892 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-tls podName:e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac nodeName:}" failed. No retries permitted until 2026-07-08 16:35:47.169872975 +0000 UTC m=+72.896797826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-tls") pod "node-exporter-gm8gv" (UID: "e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac") : secret "node-exporter-tls" not found Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669943 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-sys\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.669998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.669983 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:46.670694 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.670090 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-textfile\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.670694 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.670367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cb91da-218f-41a7-bcba-cd2eb171d744-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.670802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.670668 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.670855 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.670812 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-root\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.670987 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.670961 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.671600 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.671571 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-metrics-client-ca\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.672955 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.672919 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67cb91da-218f-41a7-bcba-cd2eb171d744-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.673068 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.673044 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/67cb91da-218f-41a7-bcba-cd2eb171d744-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.673168 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.673145 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.673370 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.673345 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.673445 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.673428 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.675269 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.675243 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:46.677798 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.677768 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42gh8\" (UniqueName: \"kubernetes.io/projected/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-kube-api-access-42gh8\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:46.681501 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.681472 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szf6x\" (UniqueName: \"kubernetes.io/projected/3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b-kube-api-access-szf6x\") pod \"kube-state-metrics-7c9dbb8b7d-w5xb8\" (UID: \"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.681660 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.681636 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrp92\" (UniqueName: \"kubernetes.io/projected/67cb91da-218f-41a7-bcba-cd2eb171d744-kube-api-access-xrp92\") pod \"openshift-state-metrics-54cf96dfcf-kb62h\" (UID: \"67cb91da-218f-41a7-bcba-cd2eb171d744\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.797471 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.797377 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" Jul 08 16:35:46.804740 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.804719 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" Jul 08 16:35:46.947765 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.947690 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8"] Jul 08 16:35:46.951435 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:46.951405 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb99dbf_145a_46c9_b75a_a6f0ecf1bf4b.slice/crio-78033a5f445061be292c91eb25c0fbbf5950cba382fde4431e70b8ad623dc911 WatchSource:0}: Error finding container 78033a5f445061be292c91eb25c0fbbf5950cba382fde4431e70b8ad623dc911: Status 404 returned error can't find the container with id 78033a5f445061be292c91eb25c0fbbf5950cba382fde4431e70b8ad623dc911 Jul 08 16:35:46.966135 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:46.966107 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h"] Jul 08 16:35:46.969288 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:46.969239 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67cb91da_218f_41a7_bcba_cd2eb171d744.slice/crio-3e72f3b27d5dc2bad4b6cf42b1d074814df335f05fd23ccbe71b52e2f3f7bbfd WatchSource:0}: Error finding container 3e72f3b27d5dc2bad4b6cf42b1d074814df335f05fd23ccbe71b52e2f3f7bbfd: Status 404 returned error can't find the container with id 3e72f3b27d5dc2bad4b6cf42b1d074814df335f05fd23ccbe71b52e2f3f7bbfd Jul 08 16:35:47.113843 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.113805 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" event={"ID":"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b","Type":"ContainerStarted","Data":"78033a5f445061be292c91eb25c0fbbf5950cba382fde4431e70b8ad623dc911"} Jul 08 16:35:47.116593 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.116344 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" event={"ID":"67cb91da-218f-41a7-bcba-cd2eb171d744","Type":"ContainerStarted","Data":"0346e6c5d6ddb91fae1804bc112aa389bdc3d1cd6521464ac8876b7bb4d7d6ae"} Jul 08 16:35:47.116820 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.116782 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" event={"ID":"67cb91da-218f-41a7-bcba-cd2eb171d744","Type":"ContainerStarted","Data":"66aba6e54de7c354d9042ea90ed85425e3c46e7cf4c840a1191454deb0649442"} Jul 08 16:35:47.116888 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.116830 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" event={"ID":"67cb91da-218f-41a7-bcba-cd2eb171d744","Type":"ContainerStarted","Data":"3e72f3b27d5dc2bad4b6cf42b1d074814df335f05fd23ccbe71b52e2f3f7bbfd"} Jul 08 16:35:47.121050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.120553 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:35:47.173590 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.173551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-tls\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:47.176091 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.176055 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac-node-exporter-tls\") pod \"node-exporter-gm8gv\" (UID: \"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac\") " pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:47.450765 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.450722 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gm8gv" Jul 08 16:35:47.462077 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:47.462044 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93a307e_1fde_466a_9f5e_ff4dc0f4e1ac.slice/crio-979f4ad20b84e202cbf7e29fd97e88afe4c2ba729fb5474a9245bc01f222825d WatchSource:0}: Error finding container 979f4ad20b84e202cbf7e29fd97e88afe4c2ba729fb5474a9245bc01f222825d: Status 404 returned error can't find the container with id 979f4ad20b84e202cbf7e29fd97e88afe4c2ba729fb5474a9245bc01f222825d Jul 08 16:35:47.559700 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.559635 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:35:47.564603 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.564579 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.567686 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.566718 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 16:35:47.567686 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567024 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 16:35:47.567686 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567108 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 16:35:47.567686 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567295 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 16:35:47.567686 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567379 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 16:35:47.567686 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567521 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 16:35:47.567686 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567587 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 16:35:47.568115 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567712 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 16:35:47.568115 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567807 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-82t7g\"" Jul 08 16:35:47.568115 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.567853 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 16:35:47.576099 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-web-config\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576212 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576114 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576212 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576212 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576151 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-config-volume\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576481 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576481 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-config-out\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576481 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576445 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576582 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576582 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576508 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.576582 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.576530 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l22rb\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-kube-api-access-l22rb\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.579025 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.578998 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:35:47.677510 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677510 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677754 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-config-volume\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677754 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677754 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677592 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677754 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677625 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677754 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677649 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677754 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677679 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-config-out\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677754 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.677754 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.678122 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.678122 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l22rb\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-kube-api-access-l22rb\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.678122 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.677831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-web-config\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.678311 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:35:47.678237 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle podName:67972630-eb8c-4635-be60-6192424add23 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:48.177927986 +0000 UTC m=+73.904852860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "67972630-eb8c-4635-be60-6192424add23") : configmap references non-existent config key: ca-bundle.crt Jul 08 16:35:47.679401 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.678726 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.679636 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.679563 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.680970 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.680817 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.680970 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.680838 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.681238 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.681123 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-web-config\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.681501 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.681484 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-config-volume\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.681699 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.681674 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.681784 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.681762 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-config-out\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.681834 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.681787 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.682255 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.682238 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.682586 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.682568 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.691100 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.691076 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l22rb\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-kube-api-access-l22rb\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:47.720353 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.720250 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-754645dd88-n5h8j"] Jul 08 16:35:47.724762 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.724740 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.734385 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.734355 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 16:35:47.734836 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.734814 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-754645dd88-n5h8j"] Jul 08 16:35:47.778212 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.778173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-oauth-config\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.778212 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.778219 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-oauth-serving-cert\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.778505 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.778242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qzzw\" (UniqueName: \"kubernetes.io/projected/45180b00-3659-4851-8be6-1f6e958a658a-kube-api-access-5qzzw\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.778505 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.778322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-serving-cert\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.778505 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.778397 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-console-config\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.778505 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.778459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-service-ca\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.778505 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.778486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-trusted-ca-bundle\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.879182 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.879142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-service-ca\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.879182 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.879187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-trusted-ca-bundle\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.879471 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.879230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-oauth-config\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.879471 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.879259 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-oauth-serving-cert\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.879471 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.879319 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qzzw\" (UniqueName: \"kubernetes.io/projected/45180b00-3659-4851-8be6-1f6e958a658a-kube-api-access-5qzzw\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.879471 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.879347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-serving-cert\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.879471 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.879413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-console-config\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.880097 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.880049 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-oauth-serving-cert\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.880097 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.880049 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-service-ca\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.880261 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.880176 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-console-config\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.880378 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.880359 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-trusted-ca-bundle\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.882880 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.882857 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-serving-cert\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.882976 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.882947 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-oauth-config\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:47.888651 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:47.888626 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qzzw\" (UniqueName: \"kubernetes.io/projected/45180b00-3659-4851-8be6-1f6e958a658a-kube-api-access-5qzzw\") pod \"console-754645dd88-n5h8j\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:48.036963 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.036878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:48.133126 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.133076 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm8gv" event={"ID":"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac","Type":"ContainerStarted","Data":"979f4ad20b84e202cbf7e29fd97e88afe4c2ba729fb5474a9245bc01f222825d"} Jul 08 16:35:48.183341 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.183294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:48.184632 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.184597 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:48.465877 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.465835 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-68f54c9d7f-wkggn"] Jul 08 16:35:48.469943 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.469903 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.475771 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.475745 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 16:35:48.476160 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.476133 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 16:35:48.476160 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.476149 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 16:35:48.476419 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.476403 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:35:48.476495 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.476425 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 16:35:48.476495 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.476439 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-k4tkn\"" Jul 08 16:35:48.476703 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.476681 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 16:35:48.479067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.478440 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e198doa19a1uj\"" Jul 08 16:35:48.486116 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-tls\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.486303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486142 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.486303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.486303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486219 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b05f903-b7be-484e-833f-6fe034fd5f90-metrics-client-ca\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.486303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt9nw\" (UniqueName: \"kubernetes.io/projected/1b05f903-b7be-484e-833f-6fe034fd5f90-kube-api-access-pt9nw\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.486303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486290 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-grpc-tls\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.486511 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486328 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.486511 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.486623 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.486598 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68f54c9d7f-wkggn"] Jul 08 16:35:48.587167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.587132 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt9nw\" (UniqueName: \"kubernetes.io/projected/1b05f903-b7be-484e-833f-6fe034fd5f90-kube-api-access-pt9nw\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.587375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.587174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-grpc-tls\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.587375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.587201 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.587375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.587218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.587375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.587267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-tls\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.587375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.587333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.587375 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.587361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.587762 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.587419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b05f903-b7be-484e-833f-6fe034fd5f90-metrics-client-ca\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.588419 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.588359 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b05f903-b7be-484e-833f-6fe034fd5f90-metrics-client-ca\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.590171 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.590136 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.590578 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.590556 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.590822 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.590799 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-tls\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.590992 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.590969 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.592021 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.592000 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-grpc-tls\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.592021 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.592014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1b05f903-b7be-484e-833f-6fe034fd5f90-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.596945 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.596918 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt9nw\" (UniqueName: \"kubernetes.io/projected/1b05f903-b7be-484e-833f-6fe034fd5f90-kube-api-access-pt9nw\") pod \"thanos-querier-68f54c9d7f-wkggn\" (UID: \"1b05f903-b7be-484e-833f-6fe034fd5f90\") " pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:48.781694 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:48.781599 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:49.142748 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:49.142537 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" event={"ID":"67cb91da-218f-41a7-bcba-cd2eb171d744","Type":"ContainerStarted","Data":"bfc2e8931a4281ee3d29ff3f6931b3771dacf7b6c6bf88a584fd78b6993d9135"} Jul 08 16:35:49.142748 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:49.142619 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68f54c9d7f-wkggn"] Jul 08 16:35:49.145226 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:49.145137 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" event={"ID":"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b","Type":"ContainerStarted","Data":"70c9d628fe630f8808f832bec1d6bed2843200a97d9d96b12783ae6c3b28d89e"} Jul 08 16:35:49.146722 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:49.146696 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b05f903_b7be_484e_833f_6fe034fd5f90.slice/crio-7c7d39f7dd537424e4ccda746e4f89b9ec61ea55c681e41b281044d71383b3ba WatchSource:0}: Error finding container 7c7d39f7dd537424e4ccda746e4f89b9ec61ea55c681e41b281044d71383b3ba: Status 404 returned error can't find the container with id 7c7d39f7dd537424e4ccda746e4f89b9ec61ea55c681e41b281044d71383b3ba Jul 08 16:35:49.148541 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:49.148459 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm8gv" event={"ID":"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac","Type":"ContainerStarted","Data":"e93bf16b33fe59b825ec22bb35071a7d2ae69a376d7fe16867cd7dbfa29acb85"} Jul 08 16:35:49.175075 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:49.174833 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kb62h" podStartSLOduration=1.321542391 podStartE2EDuration="3.174809282s" totalImagesPullingTime="1.853266891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:49.171906587 +0000 UTC m=+74.898831464" watchObservedRunningTime="2026-07-08 16:35:49.174809282 +0000 UTC m=+74.901734156" Jul 08 16:35:49.371072 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:49.370985 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-754645dd88-n5h8j"] Jul 08 16:35:49.374224 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:49.374191 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45180b00_3659_4851_8be6_1f6e958a658a.slice/crio-d7d74b17cc2108ef6b85bd0cb9f41ad776b749f6643920f2da970a7d36717aad WatchSource:0}: Error finding container d7d74b17cc2108ef6b85bd0cb9f41ad776b749f6643920f2da970a7d36717aad: Status 404 returned error can't find the container with id d7d74b17cc2108ef6b85bd0cb9f41ad776b749f6643920f2da970a7d36717aad Jul 08 16:35:49.382713 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:49.382686 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:35:49.386234 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:49.386212 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67972630_eb8c_4635_be60_6192424add23.slice/crio-73ded519d9099aed1e4666649170b350f44e2b9ad644ba8a7864d70108263931 WatchSource:0}: Error finding container 73ded519d9099aed1e4666649170b350f44e2b9ad644ba8a7864d70108263931: Status 404 returned error can't find the container with id 73ded519d9099aed1e4666649170b350f44e2b9ad644ba8a7864d70108263931 Jul 08 16:35:50.155871 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.155830 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" event={"ID":"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b","Type":"ContainerStarted","Data":"6fe539dd0c09705d2a261fe7d6d9a625d41c7b6be810ca3fc145c538fcbbf37c"} Jul 08 16:35:50.156330 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.155941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" event={"ID":"3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b","Type":"ContainerStarted","Data":"db1d2fdc46243b8e54d27b6adbe0f122a2ce3b232f69f760993ebbbb75d54e7d"} Jul 08 16:35:50.158698 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.158667 2578 generic.go:356] "Generic (PLEG): container finished" podID="e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac" containerID="e93bf16b33fe59b825ec22bb35071a7d2ae69a376d7fe16867cd7dbfa29acb85" exitCode=0 Jul 08 16:35:50.158838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.158756 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm8gv" event={"ID":"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac","Type":"ContainerDied","Data":"e93bf16b33fe59b825ec22bb35071a7d2ae69a376d7fe16867cd7dbfa29acb85"} Jul 08 16:35:50.161099 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.161073 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerStarted","Data":"73ded519d9099aed1e4666649170b350f44e2b9ad644ba8a7864d70108263931"} Jul 08 16:35:50.163168 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.163140 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754645dd88-n5h8j" event={"ID":"45180b00-3659-4851-8be6-1f6e958a658a","Type":"ContainerStarted","Data":"0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06"} Jul 08 16:35:50.163302 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.163192 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754645dd88-n5h8j" event={"ID":"45180b00-3659-4851-8be6-1f6e958a658a","Type":"ContainerStarted","Data":"d7d74b17cc2108ef6b85bd0cb9f41ad776b749f6643920f2da970a7d36717aad"} Jul 08 16:35:50.164836 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.164795 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" event={"ID":"1b05f903-b7be-484e-833f-6fe034fd5f90","Type":"ContainerStarted","Data":"7c7d39f7dd537424e4ccda746e4f89b9ec61ea55c681e41b281044d71383b3ba"} Jul 08 16:35:50.178611 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.178560 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-w5xb8" podStartSLOduration=2.17665483 podStartE2EDuration="4.178543045s" totalImagesPullingTime="2.001888215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:50.177308297 +0000 UTC m=+75.904233170" watchObservedRunningTime="2026-07-08 16:35:50.178543045 +0000 UTC m=+75.905467919" Jul 08 16:35:50.197672 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.197588 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-754645dd88-n5h8j" podStartSLOduration=3.197570237 podStartE2EDuration="3.197570237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:50.197188021 +0000 UTC m=+75.924112917" watchObservedRunningTime="2026-07-08 16:35:50.197570237 +0000 UTC m=+75.924495112" Jul 08 16:35:50.799849 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.799747 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2"] Jul 08 16:35:50.803316 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.803291 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:50.805709 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.805682 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 16:35:50.806058 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.806028 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 16:35:50.806404 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.806381 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 16:35:50.806652 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.806633 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 16:35:50.806775 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.806708 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ww5rd\"" Jul 08 16:35:50.806775 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.806762 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ekafj87ug24ea\"" Jul 08 16:35:50.812265 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.812243 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2"] Jul 08 16:35:50.911634 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.911597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-secret-metrics-server-client-certs\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:50.911831 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.911647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-secret-metrics-server-tls\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:50.911831 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.911686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/926d189c-09be-49bb-8442-95ae960ee0b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:50.911831 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.911738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn7qs\" (UniqueName: \"kubernetes.io/projected/926d189c-09be-49bb-8442-95ae960ee0b5-kube-api-access-wn7qs\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:50.911831 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.911808 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-client-ca-bundle\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:50.912040 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.911842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/926d189c-09be-49bb-8442-95ae960ee0b5-metrics-server-audit-profiles\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:50.912040 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:50.911876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/926d189c-09be-49bb-8442-95ae960ee0b5-audit-log\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-secret-metrics-server-client-certs\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013067 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013067 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-secret-metrics-server-tls\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013359 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/926d189c-09be-49bb-8442-95ae960ee0b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013359 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn7qs\" (UniqueName: \"kubernetes.io/projected/926d189c-09be-49bb-8442-95ae960ee0b5-kube-api-access-wn7qs\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013359 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013184 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-client-ca-bundle\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013509 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/926d189c-09be-49bb-8442-95ae960ee0b5-metrics-server-audit-profiles\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013509 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/926d189c-09be-49bb-8442-95ae960ee0b5-audit-log\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013854 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013828 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/926d189c-09be-49bb-8442-95ae960ee0b5-audit-log\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.013987 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.013940 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/926d189c-09be-49bb-8442-95ae960ee0b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.014355 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.014335 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/926d189c-09be-49bb-8442-95ae960ee0b5-metrics-server-audit-profiles\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.016094 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.016068 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-secret-metrics-server-client-certs\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.016355 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.016333 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-secret-metrics-server-tls\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.016355 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.016346 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/926d189c-09be-49bb-8442-95ae960ee0b5-client-ca-bundle\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.021040 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.021020 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn7qs\" (UniqueName: \"kubernetes.io/projected/926d189c-09be-49bb-8442-95ae960ee0b5-kube-api-access-wn7qs\") pod \"metrics-server-7b9b86cb7d-ncgl2\" (UID: \"926d189c-09be-49bb-8442-95ae960ee0b5\") " pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.114420 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.114255 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:35:51.173676 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.173625 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm8gv" event={"ID":"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac","Type":"ContainerStarted","Data":"363735c9a577c69adecfd9ad304f5c68b1bb733e55c722a18d9a4333f51d2406"} Jul 08 16:35:51.174071 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.173741 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm8gv" event={"ID":"e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac","Type":"ContainerStarted","Data":"e50adb500c1cec7d34ae2af0e5d0938a55f8fa8d5e7f08d78022857903760015"} Jul 08 16:35:51.175545 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.175519 2578 generic.go:356] "Generic (PLEG): container finished" podID="67972630-eb8c-4635-be60-6192424add23" containerID="60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c" exitCode=0 Jul 08 16:35:51.175684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.175604 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerDied","Data":"60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c"} Jul 08 16:35:51.196571 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.196520 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gm8gv" podStartSLOduration=3.591387478 podStartE2EDuration="5.196502819s" totalImagesPullingTime="1.492523274s" totalInitContainerRuntime="112.592067ms" isStatefulPod=true podCreationTimestamp="2026-07-08 16:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:51.195630462 +0000 UTC m=+76.922555355" watchObservedRunningTime="2026-07-08 16:35:51.196502819 +0000 UTC m=+76.923427745" Jul 08 16:35:51.262767 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.262727 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp"] Jul 08 16:35:51.268380 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.268353 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:51.270952 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.270875 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jlbj4\"" Jul 08 16:35:51.271319 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.271116 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 16:35:51.274961 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.274933 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp"] Jul 08 16:35:51.319538 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.319503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g96q5\" (UniqueName: \"kubernetes.io/projected/4c0269a7-e043-41c0-a728-62a6da22a6b2-kube-api-access-g96q5\") pod \"monitoring-plugin-d5dcff7cf-gwtkp\" (UID: \"4c0269a7-e043-41c0-a728-62a6da22a6b2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:51.319719 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.319574 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4c0269a7-e043-41c0-a728-62a6da22a6b2-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gwtkp\" (UID: \"4c0269a7-e043-41c0-a728-62a6da22a6b2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:51.349767 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.349733 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2"] Jul 08 16:35:51.352659 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:51.352634 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926d189c_09be_49bb_8442_95ae960ee0b5.slice/crio-771b5403df6a8adf0ed15a8c6b62677fb77b84fd98ec4df9070d6daebc035991 WatchSource:0}: Error finding container 771b5403df6a8adf0ed15a8c6b62677fb77b84fd98ec4df9070d6daebc035991: Status 404 returned error can't find the container with id 771b5403df6a8adf0ed15a8c6b62677fb77b84fd98ec4df9070d6daebc035991 Jul 08 16:35:51.423039 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.420583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4c0269a7-e043-41c0-a728-62a6da22a6b2-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gwtkp\" (UID: \"4c0269a7-e043-41c0-a728-62a6da22a6b2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:51.423039 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.420723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g96q5\" (UniqueName: \"kubernetes.io/projected/4c0269a7-e043-41c0-a728-62a6da22a6b2-kube-api-access-g96q5\") pod \"monitoring-plugin-d5dcff7cf-gwtkp\" (UID: \"4c0269a7-e043-41c0-a728-62a6da22a6b2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:51.423990 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.423948 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4c0269a7-e043-41c0-a728-62a6da22a6b2-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gwtkp\" (UID: \"4c0269a7-e043-41c0-a728-62a6da22a6b2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:51.430121 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.430091 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g96q5\" (UniqueName: \"kubernetes.io/projected/4c0269a7-e043-41c0-a728-62a6da22a6b2-kube-api-access-g96q5\") pod \"monitoring-plugin-d5dcff7cf-gwtkp\" (UID: \"4c0269a7-e043-41c0-a728-62a6da22a6b2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:51.581429 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.581394 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:51.732682 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:51.732603 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp"] Jul 08 16:35:51.737629 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:51.737596 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c0269a7_e043_41c0_a728_62a6da22a6b2.slice/crio-6daf3c2face63c40bcf212fe0b145f7f3be0cd17eed8922042b2164552bfbd58 WatchSource:0}: Error finding container 6daf3c2face63c40bcf212fe0b145f7f3be0cd17eed8922042b2164552bfbd58: Status 404 returned error can't find the container with id 6daf3c2face63c40bcf212fe0b145f7f3be0cd17eed8922042b2164552bfbd58 Jul 08 16:35:52.184216 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.184118 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" event={"ID":"926d189c-09be-49bb-8442-95ae960ee0b5","Type":"ContainerStarted","Data":"771b5403df6a8adf0ed15a8c6b62677fb77b84fd98ec4df9070d6daebc035991"} Jul 08 16:35:52.201167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.201124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" event={"ID":"1b05f903-b7be-484e-833f-6fe034fd5f90","Type":"ContainerStarted","Data":"208661d150149b94f75e084ecbd7734d21e801d9428501427a778895825324f8"} Jul 08 16:35:52.201434 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.201225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" event={"ID":"1b05f903-b7be-484e-833f-6fe034fd5f90","Type":"ContainerStarted","Data":"7ac4aad79f64000117ef621c66b9e8ae0551cc48eec642e9c9982060de135980"} Jul 08 16:35:52.201434 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.201299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" event={"ID":"1b05f903-b7be-484e-833f-6fe034fd5f90","Type":"ContainerStarted","Data":"dd73b6664d616c6dce13197da143cf4974dfc6fab1114691769212ea925747ab"} Jul 08 16:35:52.205519 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.205449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" event={"ID":"4c0269a7-e043-41c0-a728-62a6da22a6b2","Type":"ContainerStarted","Data":"6daf3c2face63c40bcf212fe0b145f7f3be0cd17eed8922042b2164552bfbd58"} Jul 08 16:35:52.923766 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.923730 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:35:52.928190 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.928159 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:52.933659 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.933625 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 16:35:52.933821 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.933784 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 16:35:52.933821 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.933626 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 16:35:52.933937 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.933783 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 16:35:52.934200 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934175 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 16:35:52.934200 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934189 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 16:35:52.934401 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934223 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 16:35:52.934401 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934262 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a5kivptssp17\"" Jul 08 16:35:52.934401 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934307 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 16:35:52.934401 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934347 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 16:35:52.934615 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934440 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 16:35:52.934615 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934452 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 16:35:52.935122 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.934970 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 16:35:52.935487 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.935465 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vpbcd\"" Jul 08 16:35:52.937592 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.937465 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 16:35:52.945052 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.945019 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 16:35:52.948961 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:52.948925 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:35:53.035690 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.035882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035709 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.035882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.035882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035808 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.035882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035846 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.035882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035878 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-config-out\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036141 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036141 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-web-config\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036141 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.035999 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-config\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036141 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036034 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036141 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036141 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036427 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036427 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036193 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036427 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036222 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036427 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036248 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036427 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036427 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl8zn\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-kube-api-access-wl8zn\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036427 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.036675 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.036435 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.136936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.136990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137152 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-config-out\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-web-config\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137315 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-config\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137491 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.137939 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.137575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl8zn\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-kube-api-access-wl8zn\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.139092 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.138666 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.140719 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.140426 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.141290 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.141242 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.143895 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.143818 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.144006 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.143971 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-config-out\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.145268 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.144644 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.145268 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.144974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.146182 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.145736 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.146182 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.146123 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.149746 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.149209 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.149746 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.149586 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-web-config\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.150464 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.150439 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.152789 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.152762 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.154594 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.153950 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.154594 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.154536 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.154594 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.154562 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl8zn\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-kube-api-access-wl8zn\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.154969 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.154944 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.155751 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.155684 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-config\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.157253 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.157205 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.157418 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.157328 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.225505 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.225466 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerStarted","Data":"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181"} Jul 08 16:35:53.236153 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.236074 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" event={"ID":"926d189c-09be-49bb-8442-95ae960ee0b5","Type":"ContainerStarted","Data":"9b48e5f4f4eeaa99d89fa154b8f15ea07f1f5b7de28f8f2f8c8089cdb1518adc"} Jul 08 16:35:53.245387 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.245054 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:35:53.446269 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.445056 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" podStartSLOduration=1.753465064 podStartE2EDuration="3.445031019s" totalImagesPullingTime="1.691565955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:53.262082157 +0000 UTC m=+78.989007078" watchObservedRunningTime="2026-07-08 16:35:53.445031019 +0000 UTC m=+79.171955894" Jul 08 16:35:53.446269 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:53.446178 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:35:53.706665 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:35:53.706618 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704ca4da_b93b_4a95_ad63_18cac5be6daa.slice/crio-e0074cb0dcb62187aef89105645baad9a79c5b224767380e524323eafbe98e87 WatchSource:0}: Error finding container e0074cb0dcb62187aef89105645baad9a79c5b224767380e524323eafbe98e87: Status 404 returned error can't find the container with id e0074cb0dcb62187aef89105645baad9a79c5b224767380e524323eafbe98e87 Jul 08 16:35:54.244964 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.244927 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerStarted","Data":"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398"} Jul 08 16:35:54.245369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.245015 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerStarted","Data":"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380"} Jul 08 16:35:54.245369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.245056 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerStarted","Data":"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce"} Jul 08 16:35:54.245369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.245098 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerStarted","Data":"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850"} Jul 08 16:35:54.248621 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.248585 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" event={"ID":"1b05f903-b7be-484e-833f-6fe034fd5f90","Type":"ContainerStarted","Data":"d8f1dd839f35f0489bcff6f71feb850e9b727ec57fad36637f10de73bba9480e"} Jul 08 16:35:54.248749 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.248672 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" event={"ID":"1b05f903-b7be-484e-833f-6fe034fd5f90","Type":"ContainerStarted","Data":"f52cd37bd3301bb96f5bd5e89059a3450ab07839616c275e4d41264acb099f50"} Jul 08 16:35:54.248749 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.248719 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" event={"ID":"1b05f903-b7be-484e-833f-6fe034fd5f90","Type":"ContainerStarted","Data":"812dd8a05e254336d55a2e348bf71773eee005f43cb21cb6f703fbeff843805d"} Jul 08 16:35:54.248871 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.248766 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:35:54.250388 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.250356 2578 generic.go:356] "Generic (PLEG): container finished" podID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" exitCode=0 Jul 08 16:35:54.250499 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.250421 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerDied","Data":"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8"} Jul 08 16:35:54.250499 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.250489 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerStarted","Data":"e0074cb0dcb62187aef89105645baad9a79c5b224767380e524323eafbe98e87"} Jul 08 16:35:54.252549 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.252410 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" event={"ID":"4c0269a7-e043-41c0-a728-62a6da22a6b2","Type":"ContainerStarted","Data":"12e1f465932c67d2e774a4708a86ad799b45b3acf694d75d55cd8b0bc342aac0"} Jul 08 16:35:54.273990 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.273934 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" podStartSLOduration=1.7275185400000002 podStartE2EDuration="6.27391439s" totalImagesPullingTime="4.54639585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:54.271082914 +0000 UTC m=+79.998007788" watchObservedRunningTime="2026-07-08 16:35:54.27391439 +0000 UTC m=+80.000839263" Jul 08 16:35:54.288053 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:54.288003 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" podStartSLOduration=1.127624564 podStartE2EDuration="3.287986638s" totalImagesPullingTime="2.160362074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:54.286769238 +0000 UTC m=+80.013694113" watchObservedRunningTime="2026-07-08 16:35:54.287986638 +0000 UTC m=+80.014911510" Jul 08 16:35:55.260223 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:55.260178 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerStarted","Data":"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f"} Jul 08 16:35:55.260697 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:55.260369 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:55.265693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:55.265671 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gwtkp" Jul 08 16:35:55.297086 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:55.296963 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.349675331 podStartE2EDuration="8.296939134s" totalImagesPullingTime="2.931803155s" totalInitContainerRuntime="15.460648ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:55.295295122 +0000 UTC m=+81.022219987" watchObservedRunningTime="2026-07-08 16:35:55.296939134 +0000 UTC m=+81.023864008" Jul 08 16:35:55.917829 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:55.917790 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-754645dd88-n5h8j"] Jul 08 16:35:58.038021 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:58.037992 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:35:58.280508 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:58.280476 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerStarted","Data":"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9"} Jul 08 16:35:58.280660 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:58.280544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerStarted","Data":"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649"} Jul 08 16:35:58.280660 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:58.280572 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerStarted","Data":"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1"} Jul 08 16:35:58.280660 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:58.280609 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerStarted","Data":"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291"} Jul 08 16:35:59.287697 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:59.287659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerStarted","Data":"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb"} Jul 08 16:35:59.288076 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:59.287735 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerStarted","Data":"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c"} Jul 08 16:35:59.325904 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:35:59.325845 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.631555537 podStartE2EDuration="7.325828038s" totalImagesPullingTime="3.66115058s" totalInitContainerRuntime="33.121921ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:59.324671402 +0000 UTC m=+85.051596319" watchObservedRunningTime="2026-07-08 16:35:59.325828038 +0000 UTC m=+85.052752911" Jul 08 16:36:00.268414 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:00.268388 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-68f54c9d7f-wkggn" Jul 08 16:36:03.245819 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:03.245782 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:36:07.274775 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:07.274741 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d89d49546-m8t4f"] Jul 08 16:36:11.115297 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:11.115242 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:36:11.115688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:11.115329 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:36:17.118690 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:17.118658 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-knbpm" Jul 08 16:36:20.943078 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:20.943009 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-754645dd88-n5h8j" podUID="45180b00-3659-4851-8be6-1f6e958a658a" containerName="console" containerID="cri-o://0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06" gracePeriod=15 Jul 08 16:36:21.189021 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.188997 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:36:21.294008 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.293928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-service-ca\") pod \"45180b00-3659-4851-8be6-1f6e958a658a\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " Jul 08 16:36:21.294008 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.293999 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-serving-cert\") pod \"45180b00-3659-4851-8be6-1f6e958a658a\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " Jul 08 16:36:21.294227 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294062 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-oauth-config\") pod \"45180b00-3659-4851-8be6-1f6e958a658a\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " Jul 08 16:36:21.294227 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294094 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qzzw\" (UniqueName: \"kubernetes.io/projected/45180b00-3659-4851-8be6-1f6e958a658a-kube-api-access-5qzzw\") pod \"45180b00-3659-4851-8be6-1f6e958a658a\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " Jul 08 16:36:21.294227 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294120 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-console-config\") pod \"45180b00-3659-4851-8be6-1f6e958a658a\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " Jul 08 16:36:21.294227 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294153 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-oauth-serving-cert\") pod \"45180b00-3659-4851-8be6-1f6e958a658a\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " Jul 08 16:36:21.294227 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294198 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-trusted-ca-bundle\") pod \"45180b00-3659-4851-8be6-1f6e958a658a\" (UID: \"45180b00-3659-4851-8be6-1f6e958a658a\") " Jul 08 16:36:21.294527 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294319 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-service-ca" (OuterVolumeSpecName: "service-ca") pod "45180b00-3659-4851-8be6-1f6e958a658a" (UID: "45180b00-3659-4851-8be6-1f6e958a658a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:36:21.294527 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294499 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-service-ca\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:21.294651 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294630 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "45180b00-3659-4851-8be6-1f6e958a658a" (UID: "45180b00-3659-4851-8be6-1f6e958a658a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:36:21.295048 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.294872 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "45180b00-3659-4851-8be6-1f6e958a658a" (UID: "45180b00-3659-4851-8be6-1f6e958a658a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:36:21.295212 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.295190 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-console-config" (OuterVolumeSpecName: "console-config") pod "45180b00-3659-4851-8be6-1f6e958a658a" (UID: "45180b00-3659-4851-8be6-1f6e958a658a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:36:21.296759 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.296724 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "45180b00-3659-4851-8be6-1f6e958a658a" (UID: "45180b00-3659-4851-8be6-1f6e958a658a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:36:21.296893 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.296858 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45180b00-3659-4851-8be6-1f6e958a658a-kube-api-access-5qzzw" (OuterVolumeSpecName: "kube-api-access-5qzzw") pod "45180b00-3659-4851-8be6-1f6e958a658a" (UID: "45180b00-3659-4851-8be6-1f6e958a658a"). InnerVolumeSpecName "kube-api-access-5qzzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:36:21.296981 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.296954 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "45180b00-3659-4851-8be6-1f6e958a658a" (UID: "45180b00-3659-4851-8be6-1f6e958a658a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:36:21.383121 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.383080 2578 generic.go:356] "Generic (PLEG): container finished" podID="45180b00-3659-4851-8be6-1f6e958a658a" containerID="0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06" exitCode=0 Jul 08 16:36:21.383352 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.383136 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754645dd88-n5h8j" Jul 08 16:36:21.383352 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.383138 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754645dd88-n5h8j" event={"ID":"45180b00-3659-4851-8be6-1f6e958a658a","Type":"ContainerDied","Data":"0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06"} Jul 08 16:36:21.383352 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.383263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754645dd88-n5h8j" event={"ID":"45180b00-3659-4851-8be6-1f6e958a658a","Type":"ContainerDied","Data":"d7d74b17cc2108ef6b85bd0cb9f41ad776b749f6643920f2da970a7d36717aad"} Jul 08 16:36:21.383352 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.383311 2578 scope.go:117] "RemoveContainer" containerID="0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06" Jul 08 16:36:21.392689 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.392666 2578 scope.go:117] "RemoveContainer" containerID="0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06" Jul 08 16:36:21.392982 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:36:21.392959 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06\": container with ID starting with 0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06 not found: ID does not exist" containerID="0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06" Jul 08 16:36:21.393030 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.392993 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06"} err="failed to get container status \"0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06\": rpc error: code = NotFound desc = could not find container \"0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06\": container with ID starting with 0e1a6ea22832bc01e16551eb6e4b1bf53046ffe05d090b92288a1b6c05bd0a06 not found: ID does not exist" Jul 08 16:36:21.395099 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.395080 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-serving-cert\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:21.395169 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.395101 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45180b00-3659-4851-8be6-1f6e958a658a-console-oauth-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:21.395169 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.395112 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qzzw\" (UniqueName: \"kubernetes.io/projected/45180b00-3659-4851-8be6-1f6e958a658a-kube-api-access-5qzzw\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:21.395169 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.395122 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-console-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:21.395169 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.395130 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-oauth-serving-cert\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:21.395169 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.395139 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45180b00-3659-4851-8be6-1f6e958a658a-trusted-ca-bundle\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:21.403446 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.403412 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-754645dd88-n5h8j"] Jul 08 16:36:21.406325 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:21.406300 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-754645dd88-n5h8j"] Jul 08 16:36:22.745289 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:22.745251 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45180b00-3659-4851-8be6-1f6e958a658a" path="/var/lib/kubelet/pods/45180b00-3659-4851-8be6-1f6e958a658a/volumes" Jul 08 16:36:31.120967 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:31.120918 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:36:31.125190 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:31.125168 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b9b86cb7d-ncgl2" Jul 08 16:36:32.295860 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.295793 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7d89d49546-m8t4f" podUID="f9c47575-c427-42f8-92d3-4f7986f4e618" containerName="console" containerID="cri-o://a0401fd43077a7f70d4892388a9909caf4eec82b9ba9b7d679bb4dcee9134981" gracePeriod=15 Jul 08 16:36:32.431478 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.431444 2578 generic.go:356] "Generic (PLEG): container finished" podID="f9c47575-c427-42f8-92d3-4f7986f4e618" containerID="a0401fd43077a7f70d4892388a9909caf4eec82b9ba9b7d679bb4dcee9134981" exitCode=0 Jul 08 16:36:32.431658 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.431518 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d89d49546-m8t4f" event={"ID":"f9c47575-c427-42f8-92d3-4f7986f4e618","Type":"ContainerDied","Data":"a0401fd43077a7f70d4892388a9909caf4eec82b9ba9b7d679bb4dcee9134981"} Jul 08 16:36:32.542735 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.542709 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:36:32.697560 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.697523 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-service-ca\") pod \"f9c47575-c427-42f8-92d3-4f7986f4e618\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " Jul 08 16:36:32.697560 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.697563 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-console-config\") pod \"f9c47575-c427-42f8-92d3-4f7986f4e618\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " Jul 08 16:36:32.697833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.697601 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96shx\" (UniqueName: \"kubernetes.io/projected/f9c47575-c427-42f8-92d3-4f7986f4e618-kube-api-access-96shx\") pod \"f9c47575-c427-42f8-92d3-4f7986f4e618\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " Jul 08 16:36:32.697833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.697630 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-oauth-config\") pod \"f9c47575-c427-42f8-92d3-4f7986f4e618\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " Jul 08 16:36:32.697833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.697679 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-serving-cert\") pod \"f9c47575-c427-42f8-92d3-4f7986f4e618\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " Jul 08 16:36:32.697833 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.697733 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-oauth-serving-cert\") pod \"f9c47575-c427-42f8-92d3-4f7986f4e618\" (UID: \"f9c47575-c427-42f8-92d3-4f7986f4e618\") " Jul 08 16:36:32.698034 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.697974 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-service-ca" (OuterVolumeSpecName: "service-ca") pod "f9c47575-c427-42f8-92d3-4f7986f4e618" (UID: "f9c47575-c427-42f8-92d3-4f7986f4e618"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:36:32.698090 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.698022 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-console-config" (OuterVolumeSpecName: "console-config") pod "f9c47575-c427-42f8-92d3-4f7986f4e618" (UID: "f9c47575-c427-42f8-92d3-4f7986f4e618"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:36:32.698127 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.698088 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f9c47575-c427-42f8-92d3-4f7986f4e618" (UID: "f9c47575-c427-42f8-92d3-4f7986f4e618"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:36:32.700170 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.700143 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9c47575-c427-42f8-92d3-4f7986f4e618-kube-api-access-96shx" (OuterVolumeSpecName: "kube-api-access-96shx") pod "f9c47575-c427-42f8-92d3-4f7986f4e618" (UID: "f9c47575-c427-42f8-92d3-4f7986f4e618"). InnerVolumeSpecName "kube-api-access-96shx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:36:32.700170 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.700149 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f9c47575-c427-42f8-92d3-4f7986f4e618" (UID: "f9c47575-c427-42f8-92d3-4f7986f4e618"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:36:32.700358 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.700223 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f9c47575-c427-42f8-92d3-4f7986f4e618" (UID: "f9c47575-c427-42f8-92d3-4f7986f4e618"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:36:32.798625 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.798594 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-console-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:32.798625 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.798623 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96shx\" (UniqueName: \"kubernetes.io/projected/f9c47575-c427-42f8-92d3-4f7986f4e618-kube-api-access-96shx\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:32.798625 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.798635 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-oauth-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:32.798625 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.798643 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9c47575-c427-42f8-92d3-4f7986f4e618-console-serving-cert\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:32.798892 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.798654 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-oauth-serving-cert\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:32.798892 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:32.798662 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c47575-c427-42f8-92d3-4f7986f4e618-service-ca\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:36:33.437090 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:33.437045 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d89d49546-m8t4f" event={"ID":"f9c47575-c427-42f8-92d3-4f7986f4e618","Type":"ContainerDied","Data":"2449f81ea5158cef29aab4ecd3335d2c1d28015137915a69c62f835396445137"} Jul 08 16:36:33.437611 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:33.437110 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d89d49546-m8t4f" Jul 08 16:36:33.437611 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:33.437157 2578 scope.go:117] "RemoveContainer" containerID="a0401fd43077a7f70d4892388a9909caf4eec82b9ba9b7d679bb4dcee9134981" Jul 08 16:36:33.453909 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:33.453873 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d89d49546-m8t4f"] Jul 08 16:36:33.458364 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:33.458326 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d89d49546-m8t4f"] Jul 08 16:36:34.742702 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:34.742667 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9c47575-c427-42f8-92d3-4f7986f4e618" path="/var/lib/kubelet/pods/f9c47575-c427-42f8-92d3-4f7986f4e618/volumes" Jul 08 16:36:53.246318 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:53.246253 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:36:53.267015 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:53.266991 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:36:53.538329 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:36:53.538230 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:06.934888 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:06.934852 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:37:06.935461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:06.935143 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="alertmanager" containerID="cri-o://ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181" gracePeriod=120 Jul 08 16:37:06.935461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:06.935229 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy-web" containerID="cri-o://7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce" gracePeriod=120 Jul 08 16:37:06.935461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:06.935229 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy-metric" containerID="cri-o://7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398" gracePeriod=120 Jul 08 16:37:06.935461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:06.935299 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="config-reloader" containerID="cri-o://70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850" gracePeriod=120 Jul 08 16:37:06.935461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:06.935365 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="prom-label-proxy" containerID="cri-o://8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f" gracePeriod=120 Jul 08 16:37:06.935461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:06.935328 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy" containerID="cri-o://57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380" gracePeriod=120 Jul 08 16:37:07.577649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:07.577616 2578 generic.go:356] "Generic (PLEG): container finished" podID="67972630-eb8c-4635-be60-6192424add23" containerID="8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f" exitCode=0 Jul 08 16:37:07.577649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:07.577641 2578 generic.go:356] "Generic (PLEG): container finished" podID="67972630-eb8c-4635-be60-6192424add23" containerID="57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380" exitCode=0 Jul 08 16:37:07.577649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:07.577648 2578 generic.go:356] "Generic (PLEG): container finished" podID="67972630-eb8c-4635-be60-6192424add23" containerID="70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850" exitCode=0 Jul 08 16:37:07.577649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:07.577653 2578 generic.go:356] "Generic (PLEG): container finished" podID="67972630-eb8c-4635-be60-6192424add23" containerID="ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181" exitCode=0 Jul 08 16:37:07.577926 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:07.577692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerDied","Data":"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f"} Jul 08 16:37:07.577926 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:07.577755 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerDied","Data":"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380"} Jul 08 16:37:07.577926 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:07.577782 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerDied","Data":"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850"} Jul 08 16:37:07.577926 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:07.577821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerDied","Data":"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181"} Jul 08 16:37:08.182860 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.182833 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.313998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.313904 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.313998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.313940 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-main-tls\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.313998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.313957 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-cluster-tls-config\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.313998 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.313992 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-web-config\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.314384 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314015 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-web\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.314384 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314050 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.314384 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314071 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-metric\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.314384 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314127 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-config-out\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314664 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l22rb\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-kube-api-access-l22rb\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314730 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-alertmanager-main-db\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314767 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-metrics-client-ca\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314852 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.314975 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-tls-assets\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.315017 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-config-volume\") pod \"67972630-eb8c-4635-be60-6192424add23\" (UID: \"67972630-eb8c-4635-be60-6192424add23\") " Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.315328 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.315347 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.315655 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.315374 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:08.318404 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.318346 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:08.318404 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.318364 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-kube-api-access-l22rb" (OuterVolumeSpecName: "kube-api-access-l22rb") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "kube-api-access-l22rb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:37:08.318404 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.318397 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:08.318629 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.318606 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-config-out" (OuterVolumeSpecName: "config-out") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:37:08.318688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.318639 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-config-volume" (OuterVolumeSpecName: "config-volume") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:08.318688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.318639 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:37:08.318757 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.318713 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:08.318794 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.318749 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:08.322789 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.322758 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:08.329887 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.329849 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-web-config" (OuterVolumeSpecName: "web-config") pod "67972630-eb8c-4635-be60-6192424add23" (UID: "67972630-eb8c-4635-be60-6192424add23"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:08.416522 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416473 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416522 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416515 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-main-tls\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416522 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416526 2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-cluster-tls-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416522 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416535 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-web-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416522 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416543 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416829 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416552 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416829 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416561 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-config-out\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416829 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416569 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l22rb\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-kube-api-access-l22rb\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416829 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416577 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67972630-eb8c-4635-be60-6192424add23-alertmanager-main-db\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416829 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416585 2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67972630-eb8c-4635-be60-6192424add23-metrics-client-ca\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416829 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416594 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67972630-eb8c-4635-be60-6192424add23-tls-assets\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.416829 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.416602 2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67972630-eb8c-4635-be60-6192424add23-config-volume\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:08.590189 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.590094 2578 generic.go:356] "Generic (PLEG): container finished" podID="67972630-eb8c-4635-be60-6192424add23" containerID="7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398" exitCode=0 Jul 08 16:37:08.590189 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.590123 2578 generic.go:356] "Generic (PLEG): container finished" podID="67972630-eb8c-4635-be60-6192424add23" containerID="7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce" exitCode=0 Jul 08 16:37:08.590189 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.590176 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.590189 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.590176 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerDied","Data":"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398"} Jul 08 16:37:08.590461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.590270 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerDied","Data":"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce"} Jul 08 16:37:08.590461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.590322 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67972630-eb8c-4635-be60-6192424add23","Type":"ContainerDied","Data":"73ded519d9099aed1e4666649170b350f44e2b9ad644ba8a7864d70108263931"} Jul 08 16:37:08.590461 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.590316 2578 scope.go:117] "RemoveContainer" containerID="8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f" Jul 08 16:37:08.599221 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.599202 2578 scope.go:117] "RemoveContainer" containerID="7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398" Jul 08 16:37:08.607346 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.607325 2578 scope.go:117] "RemoveContainer" containerID="57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380" Jul 08 16:37:08.614514 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.614480 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:37:08.615997 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.615976 2578 scope.go:117] "RemoveContainer" containerID="7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce" Jul 08 16:37:08.624371 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.624341 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:37:08.625123 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.625106 2578 scope.go:117] "RemoveContainer" containerID="70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850" Jul 08 16:37:08.633791 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.633770 2578 scope.go:117] "RemoveContainer" containerID="ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181" Jul 08 16:37:08.641637 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.641610 2578 scope.go:117] "RemoveContainer" containerID="60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c" Jul 08 16:37:08.650314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.650249 2578 scope.go:117] "RemoveContainer" containerID="8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f" Jul 08 16:37:08.650660 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:08.650626 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f\": container with ID starting with 8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f not found: ID does not exist" containerID="8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f" Jul 08 16:37:08.650782 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.650672 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f"} err="failed to get container status \"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f\": rpc error: code = NotFound desc = could not find container \"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f\": container with ID starting with 8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f not found: ID does not exist" Jul 08 16:37:08.650782 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.650701 2578 scope.go:117] "RemoveContainer" containerID="7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398" Jul 08 16:37:08.651031 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:08.651009 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398\": container with ID starting with 7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398 not found: ID does not exist" containerID="7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398" Jul 08 16:37:08.651090 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651035 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398"} err="failed to get container status \"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398\": rpc error: code = NotFound desc = could not find container \"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398\": container with ID starting with 7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398 not found: ID does not exist" Jul 08 16:37:08.651090 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651053 2578 scope.go:117] "RemoveContainer" containerID="57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380" Jul 08 16:37:08.651360 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:08.651340 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380\": container with ID starting with 57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380 not found: ID does not exist" containerID="57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380" Jul 08 16:37:08.651438 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651361 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:37:08.651438 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651370 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380"} err="failed to get container status \"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380\": rpc error: code = NotFound desc = could not find container \"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380\": container with ID starting with 57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380 not found: ID does not exist" Jul 08 16:37:08.651438 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651389 2578 scope.go:117] "RemoveContainer" containerID="7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce" Jul 08 16:37:08.651628 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:08.651613 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce\": container with ID starting with 7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce not found: ID does not exist" containerID="7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce" Jul 08 16:37:08.651681 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651632 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce"} err="failed to get container status \"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce\": rpc error: code = NotFound desc = could not find container \"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce\": container with ID starting with 7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce not found: ID does not exist" Jul 08 16:37:08.651681 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651646 2578 scope.go:117] "RemoveContainer" containerID="70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850" Jul 08 16:37:08.651835 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651782 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="config-reloader" Jul 08 16:37:08.651835 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651799 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="config-reloader" Jul 08 16:37:08.651835 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651819 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy-web" Jul 08 16:37:08.651835 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651827 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy-web" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651838 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651847 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:08.651848 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850\": container with ID starting with 70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850 not found: ID does not exist" containerID="70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651859 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="alertmanager" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651868 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="alertmanager" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651869 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850"} err="failed to get container status \"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850\": rpc error: code = NotFound desc = could not find container \"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850\": container with ID starting with 70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850 not found: ID does not exist" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651880 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="prom-label-proxy" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651883 2578 scope.go:117] "RemoveContainer" containerID="ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651889 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="prom-label-proxy" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651902 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45180b00-3659-4851-8be6-1f6e958a658a" containerName="console" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651911 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="45180b00-3659-4851-8be6-1f6e958a658a" containerName="console" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651931 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy-metric" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651939 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy-metric" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651954 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="init-config-reloader" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651963 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="init-config-reloader" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651975 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9c47575-c427-42f8-92d3-4f7986f4e618" containerName="console" Jul 08 16:37:08.652014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.651983 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9c47575-c427-42f8-92d3-4f7986f4e618" containerName="console" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652047 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="45180b00-3659-4851-8be6-1f6e958a658a" containerName="console" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652059 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9c47575-c427-42f8-92d3-4f7986f4e618" containerName="console" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652073 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy-metric" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652082 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="alertmanager" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652091 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy-web" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652104 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="config-reloader" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:08.652088 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181\": container with ID starting with ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181 not found: ID does not exist" containerID="ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652135 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181"} err="failed to get container status \"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181\": rpc error: code = NotFound desc = could not find container \"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181\": container with ID starting with ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181 not found: ID does not exist" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652150 2578 scope.go:117] "RemoveContainer" containerID="60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652111 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="prom-label-proxy" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652199 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="67972630-eb8c-4635-be60-6192424add23" containerName="kube-rbac-proxy" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:08.652401 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c\": container with ID starting with 60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c not found: ID does not exist" containerID="60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652418 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c"} err="failed to get container status \"60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c\": rpc error: code = NotFound desc = could not find container \"60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c\": container with ID starting with 60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c not found: ID does not exist" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652432 2578 scope.go:117] "RemoveContainer" containerID="8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652614 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f"} err="failed to get container status \"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f\": rpc error: code = NotFound desc = could not find container \"8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f\": container with ID starting with 8b549a06dfdcdaa0c30053c7f0838fc07ab8a7b1740f5365036dce8d47e8ef5f not found: ID does not exist" Jul 08 16:37:08.652628 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652630 2578 scope.go:117] "RemoveContainer" containerID="7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398" Jul 08 16:37:08.653155 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652860 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398"} err="failed to get container status \"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398\": rpc error: code = NotFound desc = could not find container \"7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398\": container with ID starting with 7ae7d71f286b685fc06b4cb8b6d56f3b55692d16ac6bf2dc48be0e77781bd398 not found: ID does not exist" Jul 08 16:37:08.653155 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.652878 2578 scope.go:117] "RemoveContainer" containerID="57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380" Jul 08 16:37:08.653155 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.653103 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380"} err="failed to get container status \"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380\": rpc error: code = NotFound desc = could not find container \"57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380\": container with ID starting with 57d6f2c4438a1761329c26d20073462ec89705beec967e49ed0ad0e197e83380 not found: ID does not exist" Jul 08 16:37:08.653155 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.653120 2578 scope.go:117] "RemoveContainer" containerID="7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce" Jul 08 16:37:08.653386 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.653369 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce"} err="failed to get container status \"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce\": rpc error: code = NotFound desc = could not find container \"7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce\": container with ID starting with 7795178cdc5d570df7153584f965282664ba16436cddd3ac56c111a9850294ce not found: ID does not exist" Jul 08 16:37:08.653431 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.653387 2578 scope.go:117] "RemoveContainer" containerID="70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850" Jul 08 16:37:08.653948 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.653924 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850"} err="failed to get container status \"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850\": rpc error: code = NotFound desc = could not find container \"70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850\": container with ID starting with 70db075ce68e08be5e0253cb1195f0ddbf35bd2f215a19f9ac879bbc3d706850 not found: ID does not exist" Jul 08 16:37:08.653948 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.653946 2578 scope.go:117] "RemoveContainer" containerID="ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181" Jul 08 16:37:08.654197 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.654179 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181"} err="failed to get container status \"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181\": rpc error: code = NotFound desc = could not find container \"ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181\": container with ID starting with ca581bec6694fc59beb836465addae4bebd1cd0e8be72679be32708859c06181 not found: ID does not exist" Jul 08 16:37:08.654267 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.654198 2578 scope.go:117] "RemoveContainer" containerID="60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c" Jul 08 16:37:08.654474 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.654446 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c"} err="failed to get container status \"60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c\": rpc error: code = NotFound desc = could not find container \"60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c\": container with ID starting with 60adb415dc98d5f7cb2136ea9ccecb802eaaddda56e597288f2eac97a37baf9c not found: ID does not exist" Jul 08 16:37:08.657668 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.657650 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.659940 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.659917 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 16:37:08.660058 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.659917 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 16:37:08.660058 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.659958 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 16:37:08.660247 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.660218 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 16:37:08.660377 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.660250 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 16:37:08.660377 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.660313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 16:37:08.660652 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.660624 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-82t7g\"" Jul 08 16:37:08.660728 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.660658 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 16:37:08.660903 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.660888 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 16:37:08.665340 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.665322 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 16:37:08.668330 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.668308 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:37:08.741221 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.741186 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67972630-eb8c-4635-be60-6192424add23" path="/var/lib/kubelet/pods/67972630-eb8c-4635-be60-6192424add23/volumes" Jul 08 16:37:08.819836 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.819804 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-config-volume\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.819836 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.819837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820061 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.819864 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820061 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.819881 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8842f9e8-41d0-474d-8522-99c3a0825ccf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820061 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.819968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820061 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.819995 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820061 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.820018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8842f9e8-41d0-474d-8522-99c3a0825ccf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820061 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.820039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-web-config\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820245 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.820065 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8842f9e8-41d0-474d-8522-99c3a0825ccf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820245 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.820099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szw2w\" (UniqueName: \"kubernetes.io/projected/8842f9e8-41d0-474d-8522-99c3a0825ccf-kube-api-access-szw2w\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820245 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.820138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8842f9e8-41d0-474d-8522-99c3a0825ccf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820245 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.820154 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8842f9e8-41d0-474d-8522-99c3a0825ccf-config-out\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.820245 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.820168 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.920689 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.920648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.920881 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.920705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.920881 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.920740 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8842f9e8-41d0-474d-8522-99c3a0825ccf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.920881 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.920768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-web-config\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.920881 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.920791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8842f9e8-41d0-474d-8522-99c3a0825ccf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921107 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.920870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szw2w\" (UniqueName: \"kubernetes.io/projected/8842f9e8-41d0-474d-8522-99c3a0825ccf-kube-api-access-szw2w\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921107 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.920945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8842f9e8-41d0-474d-8522-99c3a0825ccf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921107 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.920978 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8842f9e8-41d0-474d-8522-99c3a0825ccf-config-out\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921107 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.921007 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921107 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.921050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-config-volume\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921107 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.921081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921414 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.921171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921631 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.921538 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8842f9e8-41d0-474d-8522-99c3a0825ccf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921708 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.921691 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8842f9e8-41d0-474d-8522-99c3a0825ccf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921821 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.921768 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8842f9e8-41d0-474d-8522-99c3a0825ccf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.921917 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.921876 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8842f9e8-41d0-474d-8522-99c3a0825ccf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.924218 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.924174 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.924594 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.924571 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.924695 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.924601 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8842f9e8-41d0-474d-8522-99c3a0825ccf-config-out\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.924905 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.924854 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.925029 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.924933 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.925029 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.924995 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8842f9e8-41d0-474d-8522-99c3a0825ccf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.925029 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.925008 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-config-volume\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.925029 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.924999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.926052 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.926035 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8842f9e8-41d0-474d-8522-99c3a0825ccf-web-config\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.928535 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.928512 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szw2w\" (UniqueName: \"kubernetes.io/projected/8842f9e8-41d0-474d-8522-99c3a0825ccf-kube-api-access-szw2w\") pod \"alertmanager-main-0\" (UID: \"8842f9e8-41d0-474d-8522-99c3a0825ccf\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:08.968233 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:08.968199 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:37:09.119475 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:09.119434 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:37:09.123475 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:37:09.123433 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8842f9e8_41d0_474d_8522_99c3a0825ccf.slice/crio-8ef1bab60dcb954a2c88f4ad83f41f6fa12978c35982f345485cbcbf27dcd513 WatchSource:0}: Error finding container 8ef1bab60dcb954a2c88f4ad83f41f6fa12978c35982f345485cbcbf27dcd513: Status 404 returned error can't find the container with id 8ef1bab60dcb954a2c88f4ad83f41f6fa12978c35982f345485cbcbf27dcd513 Jul 08 16:37:09.595973 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:09.595876 2578 generic.go:356] "Generic (PLEG): container finished" podID="8842f9e8-41d0-474d-8522-99c3a0825ccf" containerID="3eb8f6184222f02a4dd00826d8c781004127fbf5b14d6b59573959ea54d58466" exitCode=0 Jul 08 16:37:09.596449 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:09.595969 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8842f9e8-41d0-474d-8522-99c3a0825ccf","Type":"ContainerDied","Data":"3eb8f6184222f02a4dd00826d8c781004127fbf5b14d6b59573959ea54d58466"} Jul 08 16:37:09.596449 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:09.596043 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8842f9e8-41d0-474d-8522-99c3a0825ccf","Type":"ContainerStarted","Data":"8ef1bab60dcb954a2c88f4ad83f41f6fa12978c35982f345485cbcbf27dcd513"} Jul 08 16:37:10.605016 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.604975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8842f9e8-41d0-474d-8522-99c3a0825ccf","Type":"ContainerStarted","Data":"a01a54d9cc466acf9d76e89fb394f945175b6e96c9601546898daeb883754a86"} Jul 08 16:37:10.605442 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.605075 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8842f9e8-41d0-474d-8522-99c3a0825ccf","Type":"ContainerStarted","Data":"b8fd555b277494792505df3201454364e096d83c524db284bd3314a427565697"} Jul 08 16:37:10.605442 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.605104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8842f9e8-41d0-474d-8522-99c3a0825ccf","Type":"ContainerStarted","Data":"e623b8c0767da9ca424a27b21d45f2618156f297f89e3c2cf204144c44d49709"} Jul 08 16:37:10.605442 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.605130 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8842f9e8-41d0-474d-8522-99c3a0825ccf","Type":"ContainerStarted","Data":"a69ae3da8b1f983fb91c4d254ba6ca3bf365109eb1d1bd25919d88a72eee0697"} Jul 08 16:37:10.605442 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.605154 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8842f9e8-41d0-474d-8522-99c3a0825ccf","Type":"ContainerStarted","Data":"2849300bd25898fcd7c88cc39456eaea0975569bb71c8f2c99158b4a2580815c"} Jul 08 16:37:10.605442 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.605178 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8842f9e8-41d0-474d-8522-99c3a0825ccf","Type":"ContainerStarted","Data":"12b45bdc95d7106430ee5fdaa5e044bf5c4fdc2e20374a7f0fbd1cebb18477d5"} Jul 08 16:37:10.632207 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.632147 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.61414501 podStartE2EDuration="2.632125104s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.980094ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:37:10.630820747 +0000 UTC m=+156.357745621" watchObservedRunningTime="2026-07-08 16:37:10.632125104 +0000 UTC m=+156.359049977" Jul 08 16:37:10.950653 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.950603 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-ccb8dd978-f4xv9"] Jul 08 16:37:10.954804 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.954769 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:10.956969 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.956940 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 16:37:10.957119 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.956940 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 16:37:10.957119 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.957065 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qxgkl\"" Jul 08 16:37:10.957238 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.956985 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 16:37:10.957238 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.956973 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 16:37:10.957238 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.957164 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 16:37:10.963436 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.963415 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 16:37:10.967023 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:10.966990 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-ccb8dd978-f4xv9"] Jul 08 16:37:11.041038 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.041000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-serving-certs-ca-bundle\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.041220 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.041042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-metrics-client-ca\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.041220 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.041101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-federate-client-tls\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.041220 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.041124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.041369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.041214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-secret-telemeter-client\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.041369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.041253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4ls4\" (UniqueName: \"kubernetes.io/projected/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-kube-api-access-g4ls4\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.041369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.041324 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.041369 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.041343 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-telemeter-client-tls\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.142542 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.142503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-serving-certs-ca-bundle\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.142542 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.142556 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-metrics-client-ca\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.142822 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.142599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-federate-client-tls\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.142822 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.142629 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.143764 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.143386 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-metrics-client-ca\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.143764 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.143443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-secret-telemeter-client\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.143764 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.143479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4ls4\" (UniqueName: \"kubernetes.io/projected/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-kube-api-access-g4ls4\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.143764 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.143511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.143764 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.143538 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-telemeter-client-tls\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.145343 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.145314 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-serving-certs-ca-bundle\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.145721 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.145695 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.147454 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.147428 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-telemeter-client-tls\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.147562 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.147544 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-federate-client-tls\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.147783 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.147760 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.147838 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.147818 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-secret-telemeter-client\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.153481 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.153450 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4ls4\" (UniqueName: \"kubernetes.io/projected/ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5-kube-api-access-g4ls4\") pod \"telemeter-client-ccb8dd978-f4xv9\" (UID: \"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5\") " pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.267035 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.266930 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" Jul 08 16:37:11.273587 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.273552 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:37:11.274209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.273927 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="prometheus" containerID="cri-o://cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" gracePeriod=600 Jul 08 16:37:11.274209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.273954 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy-thanos" containerID="cri-o://4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" gracePeriod=600 Jul 08 16:37:11.274209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.273983 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy" containerID="cri-o://c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" gracePeriod=600 Jul 08 16:37:11.274209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.273949 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy-web" containerID="cri-o://33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" gracePeriod=600 Jul 08 16:37:11.274209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.274093 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="config-reloader" containerID="cri-o://05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" gracePeriod=600 Jul 08 16:37:11.274209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.274113 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="thanos-sidecar" containerID="cri-o://7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" gracePeriod=600 Jul 08 16:37:11.443916 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.443890 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-ccb8dd978-f4xv9"] Jul 08 16:37:11.446591 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:37:11.446551 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb24d29_ac3e_4dbe_b19f_3d04a7a71af5.slice/crio-643199a49e84a968a420b1d85a5004b595a65ce6dcb819e9b02604821aff0c35 WatchSource:0}: Error finding container 643199a49e84a968a420b1d85a5004b595a65ce6dcb819e9b02604821aff0c35: Status 404 returned error can't find the container with id 643199a49e84a968a420b1d85a5004b595a65ce6dcb819e9b02604821aff0c35 Jul 08 16:37:11.547580 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.546806 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:11.620895 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.620862 2578 generic.go:356] "Generic (PLEG): container finished" podID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" exitCode=0 Jul 08 16:37:11.620895 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.620889 2578 generic.go:356] "Generic (PLEG): container finished" podID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" exitCode=0 Jul 08 16:37:11.620895 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.620896 2578 generic.go:356] "Generic (PLEG): container finished" podID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" exitCode=0 Jul 08 16:37:11.620895 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.620904 2578 generic.go:356] "Generic (PLEG): container finished" podID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" exitCode=0 Jul 08 16:37:11.620895 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.620911 2578 generic.go:356] "Generic (PLEG): container finished" podID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" exitCode=0 Jul 08 16:37:11.620895 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.620919 2578 generic.go:356] "Generic (PLEG): container finished" podID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" exitCode=0 Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.620952 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerDied","Data":"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb"} Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.621011 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.621078 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerDied","Data":"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c"} Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.621122 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerDied","Data":"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9"} Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.621159 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerDied","Data":"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649"} Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.621151 2578 scope.go:117] "RemoveContainer" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.621197 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerDied","Data":"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1"} Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.621236 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerDied","Data":"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291"} Jul 08 16:37:11.621613 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.621294 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"704ca4da-b93b-4a95-ad63-18cac5be6daa","Type":"ContainerDied","Data":"e0074cb0dcb62187aef89105645baad9a79c5b224767380e524323eafbe98e87"} Jul 08 16:37:11.622599 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.622570 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" event={"ID":"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5","Type":"ContainerStarted","Data":"643199a49e84a968a420b1d85a5004b595a65ce6dcb819e9b02604821aff0c35"} Jul 08 16:37:11.631899 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.631864 2578 scope.go:117] "RemoveContainer" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" Jul 08 16:37:11.643244 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.643221 2578 scope.go:117] "RemoveContainer" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" Jul 08 16:37:11.648601 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648571 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-db\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.648601 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648603 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl8zn\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-kube-api-access-wl8zn\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.648795 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648635 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.648795 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648676 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-web-config\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.648795 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648701 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-1\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.648795 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648716 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-grpc-tls\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.648795 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648751 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-metrics-client-ca\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.648795 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648780 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-tls\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648803 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-tls-assets\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648855 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648891 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-0\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648916 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-2\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648954 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-serving-certs-ca-bundle\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.648985 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-trusted-ca-bundle\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.649029 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-config-out\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.649062 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-metrics-client-certs\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649479 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.649099 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-kubelet-serving-ca-bundle\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649479 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.649114 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:11.649479 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.649130 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-kube-rbac-proxy\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649479 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.649196 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-config\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649479 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.649225 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-thanos-prometheus-http-client-file\") pod \"704ca4da-b93b-4a95-ad63-18cac5be6daa\" (UID: \"704ca4da-b93b-4a95-ad63-18cac5be6daa\") " Jul 08 16:37:11.649949 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.649543 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.650153 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.650126 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:37:11.650335 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.650307 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:11.650423 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.650397 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:11.650743 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.650686 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:11.651736 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.651680 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:11.652236 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.651950 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:11.652236 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.652000 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-kube-api-access-wl8zn" (OuterVolumeSpecName: "kube-api-access-wl8zn") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "kube-api-access-wl8zn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:37:11.652627 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.652478 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.653117 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.653089 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:37:11.653415 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.653378 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.653671 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.653610 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-config" (OuterVolumeSpecName: "config") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.654433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.654179 2578 scope.go:117] "RemoveContainer" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" Jul 08 16:37:11.654433 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.654401 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.654775 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.654744 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.654856 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.654797 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.655017 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.654995 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:37:11.655303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.655249 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-config-out" (OuterVolumeSpecName: "config-out") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:37:11.655303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.655266 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.655443 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.655394 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.663120 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.663098 2578 scope.go:117] "RemoveContainer" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" Jul 08 16:37:11.666745 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.666711 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-web-config" (OuterVolumeSpecName: "web-config") pod "704ca4da-b93b-4a95-ad63-18cac5be6daa" (UID: "704ca4da-b93b-4a95-ad63-18cac5be6daa"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:37:11.671290 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.671257 2578 scope.go:117] "RemoveContainer" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" Jul 08 16:37:11.679923 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.679899 2578 scope.go:117] "RemoveContainer" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" Jul 08 16:37:11.687572 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.687551 2578 scope.go:117] "RemoveContainer" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" Jul 08 16:37:11.687911 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:11.687889 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": container with ID starting with 4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb not found: ID does not exist" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" Jul 08 16:37:11.688013 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.687924 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb"} err="failed to get container status \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": rpc error: code = NotFound desc = could not find container \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": container with ID starting with 4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb not found: ID does not exist" Jul 08 16:37:11.688013 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.687945 2578 scope.go:117] "RemoveContainer" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" Jul 08 16:37:11.688224 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:11.688206 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": container with ID starting with c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c not found: ID does not exist" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" Jul 08 16:37:11.688270 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.688230 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c"} err="failed to get container status \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": rpc error: code = NotFound desc = could not find container \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": container with ID starting with c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c not found: ID does not exist" Jul 08 16:37:11.688270 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.688246 2578 scope.go:117] "RemoveContainer" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" Jul 08 16:37:11.688505 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:11.688487 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": container with ID starting with 33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9 not found: ID does not exist" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" Jul 08 16:37:11.688543 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.688513 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9"} err="failed to get container status \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": rpc error: code = NotFound desc = could not find container \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": container with ID starting with 33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9 not found: ID does not exist" Jul 08 16:37:11.688543 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.688529 2578 scope.go:117] "RemoveContainer" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" Jul 08 16:37:11.688733 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:11.688718 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": container with ID starting with 7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649 not found: ID does not exist" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" Jul 08 16:37:11.688773 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.688739 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649"} err="failed to get container status \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": rpc error: code = NotFound desc = could not find container \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": container with ID starting with 7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649 not found: ID does not exist" Jul 08 16:37:11.688773 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.688754 2578 scope.go:117] "RemoveContainer" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" Jul 08 16:37:11.688985 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:11.688972 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": container with ID starting with 05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1 not found: ID does not exist" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" Jul 08 16:37:11.689024 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.688987 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1"} err="failed to get container status \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": rpc error: code = NotFound desc = could not find container \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": container with ID starting with 05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1 not found: ID does not exist" Jul 08 16:37:11.689024 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.689001 2578 scope.go:117] "RemoveContainer" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" Jul 08 16:37:11.689226 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:11.689210 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": container with ID starting with cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291 not found: ID does not exist" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" Jul 08 16:37:11.689266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.689230 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291"} err="failed to get container status \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": rpc error: code = NotFound desc = could not find container \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": container with ID starting with cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291 not found: ID does not exist" Jul 08 16:37:11.689266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.689243 2578 scope.go:117] "RemoveContainer" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" Jul 08 16:37:11.689592 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:37:11.689566 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": container with ID starting with d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8 not found: ID does not exist" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" Jul 08 16:37:11.689592 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.689585 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8"} err="failed to get container status \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": rpc error: code = NotFound desc = could not find container \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": container with ID starting with d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8 not found: ID does not exist" Jul 08 16:37:11.689714 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.689602 2578 scope.go:117] "RemoveContainer" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" Jul 08 16:37:11.689839 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.689814 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb"} err="failed to get container status \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": rpc error: code = NotFound desc = could not find container \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": container with ID starting with 4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb not found: ID does not exist" Jul 08 16:37:11.689876 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.689840 2578 scope.go:117] "RemoveContainer" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" Jul 08 16:37:11.690095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.690078 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c"} err="failed to get container status \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": rpc error: code = NotFound desc = could not find container \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": container with ID starting with c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c not found: ID does not exist" Jul 08 16:37:11.690095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.690094 2578 scope.go:117] "RemoveContainer" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" Jul 08 16:37:11.690353 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.690335 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9"} err="failed to get container status \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": rpc error: code = NotFound desc = could not find container \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": container with ID starting with 33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9 not found: ID does not exist" Jul 08 16:37:11.690353 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.690353 2578 scope.go:117] "RemoveContainer" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" Jul 08 16:37:11.690589 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.690574 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649"} err="failed to get container status \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": rpc error: code = NotFound desc = could not find container \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": container with ID starting with 7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649 not found: ID does not exist" Jul 08 16:37:11.690589 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.690589 2578 scope.go:117] "RemoveContainer" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" Jul 08 16:37:11.690802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.690784 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1"} err="failed to get container status \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": rpc error: code = NotFound desc = could not find container \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": container with ID starting with 05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1 not found: ID does not exist" Jul 08 16:37:11.690849 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.690803 2578 scope.go:117] "RemoveContainer" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" Jul 08 16:37:11.691053 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691035 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291"} err="failed to get container status \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": rpc error: code = NotFound desc = could not find container \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": container with ID starting with cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291 not found: ID does not exist" Jul 08 16:37:11.691118 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691057 2578 scope.go:117] "RemoveContainer" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" Jul 08 16:37:11.691301 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691259 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8"} err="failed to get container status \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": rpc error: code = NotFound desc = could not find container \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": container with ID starting with d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8 not found: ID does not exist" Jul 08 16:37:11.691343 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691306 2578 scope.go:117] "RemoveContainer" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" Jul 08 16:37:11.691534 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691514 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb"} err="failed to get container status \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": rpc error: code = NotFound desc = could not find container \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": container with ID starting with 4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb not found: ID does not exist" Jul 08 16:37:11.691583 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691535 2578 scope.go:117] "RemoveContainer" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" Jul 08 16:37:11.691722 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691708 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c"} err="failed to get container status \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": rpc error: code = NotFound desc = could not find container \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": container with ID starting with c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c not found: ID does not exist" Jul 08 16:37:11.691767 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691722 2578 scope.go:117] "RemoveContainer" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" Jul 08 16:37:11.691951 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691936 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9"} err="failed to get container status \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": rpc error: code = NotFound desc = could not find container \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": container with ID starting with 33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9 not found: ID does not exist" Jul 08 16:37:11.691951 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.691950 2578 scope.go:117] "RemoveContainer" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" Jul 08 16:37:11.692126 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692112 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649"} err="failed to get container status \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": rpc error: code = NotFound desc = could not find container \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": container with ID starting with 7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649 not found: ID does not exist" Jul 08 16:37:11.692126 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692125 2578 scope.go:117] "RemoveContainer" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" Jul 08 16:37:11.692344 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692326 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1"} err="failed to get container status \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": rpc error: code = NotFound desc = could not find container \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": container with ID starting with 05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1 not found: ID does not exist" Jul 08 16:37:11.692397 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692346 2578 scope.go:117] "RemoveContainer" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" Jul 08 16:37:11.692531 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692505 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291"} err="failed to get container status \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": rpc error: code = NotFound desc = could not find container \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": container with ID starting with cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291 not found: ID does not exist" Jul 08 16:37:11.692531 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692526 2578 scope.go:117] "RemoveContainer" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" Jul 08 16:37:11.692745 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692726 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8"} err="failed to get container status \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": rpc error: code = NotFound desc = could not find container \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": container with ID starting with d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8 not found: ID does not exist" Jul 08 16:37:11.692788 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692746 2578 scope.go:117] "RemoveContainer" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" Jul 08 16:37:11.692967 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692948 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb"} err="failed to get container status \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": rpc error: code = NotFound desc = could not find container \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": container with ID starting with 4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb not found: ID does not exist" Jul 08 16:37:11.693005 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.692969 2578 scope.go:117] "RemoveContainer" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" Jul 08 16:37:11.693157 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.693141 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c"} err="failed to get container status \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": rpc error: code = NotFound desc = could not find container \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": container with ID starting with c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c not found: ID does not exist" Jul 08 16:37:11.693204 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.693157 2578 scope.go:117] "RemoveContainer" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" Jul 08 16:37:11.693460 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.693440 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9"} err="failed to get container status \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": rpc error: code = NotFound desc = could not find container \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": container with ID starting with 33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9 not found: ID does not exist" Jul 08 16:37:11.693460 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.693460 2578 scope.go:117] "RemoveContainer" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" Jul 08 16:37:11.693666 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.693648 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649"} err="failed to get container status \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": rpc error: code = NotFound desc = could not find container \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": container with ID starting with 7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649 not found: ID does not exist" Jul 08 16:37:11.693709 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.693670 2578 scope.go:117] "RemoveContainer" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" Jul 08 16:37:11.693884 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.693869 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1"} err="failed to get container status \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": rpc error: code = NotFound desc = could not find container \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": container with ID starting with 05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1 not found: ID does not exist" Jul 08 16:37:11.693926 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.693884 2578 scope.go:117] "RemoveContainer" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" Jul 08 16:37:11.694072 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694055 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291"} err="failed to get container status \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": rpc error: code = NotFound desc = could not find container \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": container with ID starting with cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291 not found: ID does not exist" Jul 08 16:37:11.694072 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694070 2578 scope.go:117] "RemoveContainer" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" Jul 08 16:37:11.694263 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694248 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8"} err="failed to get container status \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": rpc error: code = NotFound desc = could not find container \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": container with ID starting with d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8 not found: ID does not exist" Jul 08 16:37:11.694325 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694263 2578 scope.go:117] "RemoveContainer" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" Jul 08 16:37:11.694492 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694472 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb"} err="failed to get container status \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": rpc error: code = NotFound desc = could not find container \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": container with ID starting with 4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb not found: ID does not exist" Jul 08 16:37:11.694539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694493 2578 scope.go:117] "RemoveContainer" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" Jul 08 16:37:11.694713 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694696 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c"} err="failed to get container status \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": rpc error: code = NotFound desc = could not find container \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": container with ID starting with c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c not found: ID does not exist" Jul 08 16:37:11.694761 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694714 2578 scope.go:117] "RemoveContainer" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" Jul 08 16:37:11.694882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694865 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9"} err="failed to get container status \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": rpc error: code = NotFound desc = could not find container \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": container with ID starting with 33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9 not found: ID does not exist" Jul 08 16:37:11.694926 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.694883 2578 scope.go:117] "RemoveContainer" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" Jul 08 16:37:11.695054 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695034 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649"} err="failed to get container status \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": rpc error: code = NotFound desc = could not find container \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": container with ID starting with 7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649 not found: ID does not exist" Jul 08 16:37:11.695093 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695055 2578 scope.go:117] "RemoveContainer" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" Jul 08 16:37:11.695313 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695297 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1"} err="failed to get container status \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": rpc error: code = NotFound desc = could not find container \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": container with ID starting with 05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1 not found: ID does not exist" Jul 08 16:37:11.695313 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695312 2578 scope.go:117] "RemoveContainer" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" Jul 08 16:37:11.695512 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695494 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291"} err="failed to get container status \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": rpc error: code = NotFound desc = could not find container \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": container with ID starting with cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291 not found: ID does not exist" Jul 08 16:37:11.695512 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695512 2578 scope.go:117] "RemoveContainer" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" Jul 08 16:37:11.695743 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695728 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8"} err="failed to get container status \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": rpc error: code = NotFound desc = could not find container \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": container with ID starting with d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8 not found: ID does not exist" Jul 08 16:37:11.695787 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695744 2578 scope.go:117] "RemoveContainer" containerID="4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb" Jul 08 16:37:11.695931 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695918 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb"} err="failed to get container status \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": rpc error: code = NotFound desc = could not find container \"4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb\": container with ID starting with 4b25046aee71be83820a8e8761ccd8b9d3de0476f70f5ba2be2e9fe3b95230cb not found: ID does not exist" Jul 08 16:37:11.695931 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.695931 2578 scope.go:117] "RemoveContainer" containerID="c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c" Jul 08 16:37:11.696119 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.696102 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c"} err="failed to get container status \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": rpc error: code = NotFound desc = could not find container \"c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c\": container with ID starting with c5a0af6ba7ec8106b187ae464af01a4949cd34d39ad0c44e7985d2a349ce965c not found: ID does not exist" Jul 08 16:37:11.696157 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.696120 2578 scope.go:117] "RemoveContainer" containerID="33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9" Jul 08 16:37:11.696365 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.696345 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9"} err="failed to get container status \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": rpc error: code = NotFound desc = could not find container \"33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9\": container with ID starting with 33aff79813f49b924bd0122e8bbb55e021116d9c6be34c4ae54e0ba1a65e2ea9 not found: ID does not exist" Jul 08 16:37:11.696460 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.696368 2578 scope.go:117] "RemoveContainer" containerID="7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649" Jul 08 16:37:11.696621 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.696603 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649"} err="failed to get container status \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": rpc error: code = NotFound desc = could not find container \"7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649\": container with ID starting with 7a46695abfa5e73c0f3fec924de3bd67219736b82dfa957a91686eb9b8104649 not found: ID does not exist" Jul 08 16:37:11.696682 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.696624 2578 scope.go:117] "RemoveContainer" containerID="05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1" Jul 08 16:37:11.696857 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.696839 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1"} err="failed to get container status \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": rpc error: code = NotFound desc = could not find container \"05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1\": container with ID starting with 05e8ca4d1e0d43ae3cc0d5afdeb28287cf2d6375dbcd8ad590a4b99e33bdf6c1 not found: ID does not exist" Jul 08 16:37:11.696990 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.696861 2578 scope.go:117] "RemoveContainer" containerID="cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291" Jul 08 16:37:11.697099 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.697080 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291"} err="failed to get container status \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": rpc error: code = NotFound desc = could not find container \"cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291\": container with ID starting with cd20109f35227995b7c0f3f2cfb5061c1391b158b5b32306b6a0a149fd16d291 not found: ID does not exist" Jul 08 16:37:11.697149 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.697101 2578 scope.go:117] "RemoveContainer" containerID="d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8" Jul 08 16:37:11.697347 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.697330 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8"} err="failed to get container status \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": rpc error: code = NotFound desc = could not find container \"d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8\": container with ID starting with d88f347b554d0396b0179e90030873265c5021aef823f9c13cf56c6d0dfebde8 not found: ID does not exist" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750325 2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-grpc-tls\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750364 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-metrics-client-ca\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750379 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750409 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-tls-assets\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750424 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750435 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750445 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750457 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750471 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750485 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-config-out\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750498 2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-metrics-client-certs\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750512 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704ca4da-b93b-4a95-ad63-18cac5be6daa-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750525 2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-kube-rbac-proxy\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750538 2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.750550 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750558 2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.751110 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750572 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/704ca4da-b93b-4a95-ad63-18cac5be6daa-prometheus-k8s-db\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.751110 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750587 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl8zn\" (UniqueName: \"kubernetes.io/projected/704ca4da-b93b-4a95-ad63-18cac5be6daa-kube-api-access-wl8zn\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.751110 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750602 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.751110 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.750617 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/704ca4da-b93b-4a95-ad63-18cac5be6daa-web-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:37:11.944944 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.944910 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:37:11.948727 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.948700 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:37:11.984170 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984071 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:37:11.984592 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984567 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy-web" Jul 08 16:37:11.984688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984614 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy-web" Jul 08 16:37:11.984688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984629 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="prometheus" Jul 08 16:37:11.984688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984651 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="prometheus" Jul 08 16:37:11.984688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984666 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="init-config-reloader" Jul 08 16:37:11.984688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984674 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="init-config-reloader" Jul 08 16:37:11.984688 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984688 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984697 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984725 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="thanos-sidecar" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984733 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="thanos-sidecar" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984745 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="config-reloader" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984753 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="config-reloader" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984765 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy-thanos" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984784 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy-thanos" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984883 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="thanos-sidecar" Jul 08 16:37:11.984897 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984897 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy-thanos" Jul 08 16:37:11.985166 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984909 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="prometheus" Jul 08 16:37:11.985166 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984917 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy" Jul 08 16:37:11.985166 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984925 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="config-reloader" Jul 08 16:37:11.985166 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.984931 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" containerName="kube-rbac-proxy-web" Jul 08 16:37:11.990572 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.990549 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:11.997588 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.997299 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 16:37:11.997588 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.997500 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 16:37:11.998217 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:11.998184 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 16:37:12.000756 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000368 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 16:37:12.000756 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000390 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 16:37:12.000756 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000418 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 16:37:12.000756 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000368 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vpbcd\"" Jul 08 16:37:12.000756 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000603 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 16:37:12.000756 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000638 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 16:37:12.000756 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000651 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a5kivptssp17\"" Jul 08 16:37:12.001149 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000780 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 16:37:12.001149 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000852 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 16:37:12.001149 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.000605 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 16:37:12.001373 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.001358 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 16:37:12.002193 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.002173 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 16:37:12.005363 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.005341 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 16:37:12.010077 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.010044 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:37:12.154379 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154519 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154519 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154422 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-config\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154519 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154453 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154519 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154519 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154812 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154812 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154812 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154812 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154624 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154812 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wplql\" (UniqueName: \"kubernetes.io/projected/cc2b1371-625d-443a-b632-aa0d633db569-kube-api-access-wplql\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154812 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.154812 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154809 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.155175 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.155175 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc2b1371-625d-443a-b632-aa0d633db569-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.155175 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-web-config\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.155175 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.155175 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc2b1371-625d-443a-b632-aa0d633db569-config-out\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.155175 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.154992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.155175 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.155025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.255967 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.255870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.255967 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.255907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.255967 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.255930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.255967 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.255958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.255980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.255999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wplql\" (UniqueName: \"kubernetes.io/projected/cc2b1371-625d-443a-b632-aa0d633db569-kube-api-access-wplql\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc2b1371-625d-443a-b632-aa0d633db569-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256334 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256334 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-web-config\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256374 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc2b1371-625d-443a-b632-aa0d633db569-config-out\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-config\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256597 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.256693 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256685 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.257398 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.256939 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.257398 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.257048 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.257504 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.257424 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.257646 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.257622 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.259353 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.259330 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc2b1371-625d-443a-b632-aa0d633db569-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.259489 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.259334 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.259559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.259484 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.259621 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.259557 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.259976 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.259917 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.260131 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.260117 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.260239 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.260211 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.260326 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.260252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-web-config\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.260475 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.260422 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.261150 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.261130 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.262118 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.262096 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc2b1371-625d-443a-b632-aa0d633db569-config-out\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.262212 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.262129 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.262594 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.262577 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cc2b1371-625d-443a-b632-aa0d633db569-config\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.263352 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.263331 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cc2b1371-625d-443a-b632-aa0d633db569-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.265681 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.265659 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wplql\" (UniqueName: \"kubernetes.io/projected/cc2b1371-625d-443a-b632-aa0d633db569-kube-api-access-wplql\") pod \"prometheus-k8s-0\" (UID: \"cc2b1371-625d-443a-b632-aa0d633db569\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.301706 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.301663 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:37:12.482246 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.482218 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:37:12.483997 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:37:12.483971 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc2b1371_625d_443a_b632_aa0d633db569.slice/crio-f6792a2e621e87b17fc8821bab678b03e02db5d3b28d6d546dfc91f38e45dd6e WatchSource:0}: Error finding container f6792a2e621e87b17fc8821bab678b03e02db5d3b28d6d546dfc91f38e45dd6e: Status 404 returned error can't find the container with id f6792a2e621e87b17fc8821bab678b03e02db5d3b28d6d546dfc91f38e45dd6e Jul 08 16:37:12.631095 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.631056 2578 generic.go:356] "Generic (PLEG): container finished" podID="cc2b1371-625d-443a-b632-aa0d633db569" containerID="10c0a444b30a16d87594fff5b00c298be96fb180ee14dbabdce30c0e67c2c9d0" exitCode=0 Jul 08 16:37:12.631580 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.631119 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc2b1371-625d-443a-b632-aa0d633db569","Type":"ContainerDied","Data":"10c0a444b30a16d87594fff5b00c298be96fb180ee14dbabdce30c0e67c2c9d0"} Jul 08 16:37:12.631580 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.631206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc2b1371-625d-443a-b632-aa0d633db569","Type":"ContainerStarted","Data":"f6792a2e621e87b17fc8821bab678b03e02db5d3b28d6d546dfc91f38e45dd6e"} Jul 08 16:37:12.751624 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:12.751586 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="704ca4da-b93b-4a95-ad63-18cac5be6daa" path="/var/lib/kubelet/pods/704ca4da-b93b-4a95-ad63-18cac5be6daa/volumes" Jul 08 16:37:13.640168 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:13.640124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc2b1371-625d-443a-b632-aa0d633db569","Type":"ContainerStarted","Data":"7f604dbae8bf6c31c185d86028559883aab01ad55c45520421aea53df82b4771"} Jul 08 16:37:13.640620 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:13.640211 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc2b1371-625d-443a-b632-aa0d633db569","Type":"ContainerStarted","Data":"951eb7c5409a9d40df72d7c6f33e33788da5434ea0a0f42e3d0c6c2ff8df6181"} Jul 08 16:37:13.640620 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:13.640253 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc2b1371-625d-443a-b632-aa0d633db569","Type":"ContainerStarted","Data":"79a9c0bbee5a0817efa244adc1ade8ea8e4b5be2ad10c7afd3fbcd00e2a89aef"} Jul 08 16:37:13.640620 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:13.640321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc2b1371-625d-443a-b632-aa0d633db569","Type":"ContainerStarted","Data":"491c6714be3c0089ac9ba576c3ad7713201491302a46b3cf4c3386b74f97861d"} Jul 08 16:37:13.640620 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:13.640363 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc2b1371-625d-443a-b632-aa0d633db569","Type":"ContainerStarted","Data":"ce9b104fd9cf6a7211d3c1ec64acd2bed4fb294ea3d60e9e5264fad629023a04"} Jul 08 16:37:13.640620 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:13.640400 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cc2b1371-625d-443a-b632-aa0d633db569","Type":"ContainerStarted","Data":"f5770aab963a73e2ce1ccabc76da962ac538ed96b5116b3ddd588cf54e375f93"} Jul 08 16:37:13.673641 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:13.671886 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.646542111 podStartE2EDuration="2.671829148s" totalImagesPullingTime="0s" totalInitContainerRuntime="25.287037ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:37:13.667556634 +0000 UTC m=+159.394481508" watchObservedRunningTime="2026-07-08 16:37:13.671829148 +0000 UTC m=+159.398754022" Jul 08 16:37:14.646504 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:14.646460 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" event={"ID":"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5","Type":"ContainerStarted","Data":"ef502091cfab52fe2a18387ad809f436c8bc6847fa50f9d59a8db311c6c718fb"} Jul 08 16:37:14.646889 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:14.646533 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" event={"ID":"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5","Type":"ContainerStarted","Data":"ded427d0a188af8aef9808f9fc30273c84e7cf76092d599b6972b4c37ee258c7"} Jul 08 16:37:14.646889 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:14.646556 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" event={"ID":"ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5","Type":"ContainerStarted","Data":"36984260c21b78af17b37ae6ee9e9178b5a69ac65c76d25f9255575987e7938b"} Jul 08 16:37:14.670955 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:14.670866 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-ccb8dd978-f4xv9" podStartSLOduration=2.304917787 podStartE2EDuration="4.670847793s" totalImagesPullingTime="2.365930006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:37:14.670635325 +0000 UTC m=+160.397560202" watchObservedRunningTime="2026-07-08 16:37:14.670847793 +0000 UTC m=+160.397772666" Jul 08 16:37:17.302719 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:37:17.302673 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:38:12.302859 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:38:12.302821 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:38:12.319445 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:38:12.319407 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:38:12.910908 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:38:12.910881 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:39:34.627010 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:39:34.626981 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 16:43:20.568514 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.568479 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-4f7nw"] Jul 08 16:43:20.571441 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.571415 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:20.573865 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.573836 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 08 16:43:20.574059 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.573904 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 08 16:43:20.574409 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.574392 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-jbpxq\"" Jul 08 16:43:20.576893 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.576875 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 08 16:43:20.584187 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.584162 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-4f7nw"] Jul 08 16:43:20.645685 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.645646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert\") pod \"kserve-controller-manager-c8c545f5f-4f7nw\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:20.645685 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.645687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlsgp\" (UniqueName: \"kubernetes.io/projected/7ff24439-bb83-45d5-bbc5-3f08c972ca58-kube-api-access-qlsgp\") pod \"kserve-controller-manager-c8c545f5f-4f7nw\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:20.747054 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.747019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlsgp\" (UniqueName: \"kubernetes.io/projected/7ff24439-bb83-45d5-bbc5-3f08c972ca58-kube-api-access-qlsgp\") pod \"kserve-controller-manager-c8c545f5f-4f7nw\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:20.747240 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.747125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert\") pod \"kserve-controller-manager-c8c545f5f-4f7nw\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:20.747240 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:43:20.747217 2578 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 08 16:43:20.747378 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:43:20.747296 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert podName:7ff24439-bb83-45d5-bbc5-3f08c972ca58 nodeName:}" failed. No retries permitted until 2026-07-08 16:43:21.247257208 +0000 UTC m=+526.974182058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert") pod "kserve-controller-manager-c8c545f5f-4f7nw" (UID: "7ff24439-bb83-45d5-bbc5-3f08c972ca58") : secret "kserve-webhook-server-cert" not found Jul 08 16:43:20.760403 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:20.760367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlsgp\" (UniqueName: \"kubernetes.io/projected/7ff24439-bb83-45d5-bbc5-3f08c972ca58-kube-api-access-qlsgp\") pod \"kserve-controller-manager-c8c545f5f-4f7nw\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:21.253397 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:21.253357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert\") pod \"kserve-controller-manager-c8c545f5f-4f7nw\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:21.256043 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:21.256006 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert\") pod \"kserve-controller-manager-c8c545f5f-4f7nw\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:21.483863 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:21.483825 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:21.631524 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:21.631493 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-4f7nw"] Jul 08 16:43:21.635086 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:43:21.635052 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff24439_bb83_45d5_bbc5_3f08c972ca58.slice/crio-31f9965ee9ef4f66ebdfa25f561be04bf419f897ad8e9244d4d67ec85ec872f6 WatchSource:0}: Error finding container 31f9965ee9ef4f66ebdfa25f561be04bf419f897ad8e9244d4d67ec85ec872f6: Status 404 returned error can't find the container with id 31f9965ee9ef4f66ebdfa25f561be04bf419f897ad8e9244d4d67ec85ec872f6 Jul 08 16:43:21.636451 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:21.636426 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 16:43:22.148023 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:22.147981 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" event={"ID":"7ff24439-bb83-45d5-bbc5-3f08c972ca58","Type":"ContainerStarted","Data":"31f9965ee9ef4f66ebdfa25f561be04bf419f897ad8e9244d4d67ec85ec872f6"} Jul 08 16:43:25.165155 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:25.165120 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" event={"ID":"7ff24439-bb83-45d5-bbc5-3f08c972ca58","Type":"ContainerStarted","Data":"146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239"} Jul 08 16:43:25.165614 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:25.165243 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:25.181014 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:25.180948 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" podStartSLOduration=2.505389265 podStartE2EDuration="5.180925614s" totalImagesPullingTime="2.675536349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:43:25.179601768 +0000 UTC m=+530.906526641" watchObservedRunningTime="2026-07-08 16:43:25.180925614 +0000 UTC m=+530.907850489" Jul 08 16:43:56.174345 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.174313 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:56.335568 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.335530 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-4f7nw"] Jul 08 16:43:56.335760 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.335707 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" podUID="7ff24439-bb83-45d5-bbc5-3f08c972ca58" containerName="manager" containerID="cri-o://146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239" gracePeriod=10 Jul 08 16:43:56.365152 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.362676 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-pmx8q"] Jul 08 16:43:56.367238 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.367214 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:56.374466 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.374439 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-pmx8q"] Jul 08 16:43:56.473256 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.473219 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cccff085-ade7-4951-8fe1-2669d818c070-cert\") pod \"kserve-controller-manager-c8c545f5f-pmx8q\" (UID: \"cccff085-ade7-4951-8fe1-2669d818c070\") " pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:56.473450 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.473362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdfsq\" (UniqueName: \"kubernetes.io/projected/cccff085-ade7-4951-8fe1-2669d818c070-kube-api-access-cdfsq\") pod \"kserve-controller-manager-c8c545f5f-pmx8q\" (UID: \"cccff085-ade7-4951-8fe1-2669d818c070\") " pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:56.574576 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.574546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cccff085-ade7-4951-8fe1-2669d818c070-cert\") pod \"kserve-controller-manager-c8c545f5f-pmx8q\" (UID: \"cccff085-ade7-4951-8fe1-2669d818c070\") " pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:56.574722 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.574625 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdfsq\" (UniqueName: \"kubernetes.io/projected/cccff085-ade7-4951-8fe1-2669d818c070-kube-api-access-cdfsq\") pod \"kserve-controller-manager-c8c545f5f-pmx8q\" (UID: \"cccff085-ade7-4951-8fe1-2669d818c070\") " pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:56.577209 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.577184 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cccff085-ade7-4951-8fe1-2669d818c070-cert\") pod \"kserve-controller-manager-c8c545f5f-pmx8q\" (UID: \"cccff085-ade7-4951-8fe1-2669d818c070\") " pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:56.582870 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.582839 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdfsq\" (UniqueName: \"kubernetes.io/projected/cccff085-ade7-4951-8fe1-2669d818c070-kube-api-access-cdfsq\") pod \"kserve-controller-manager-c8c545f5f-pmx8q\" (UID: \"cccff085-ade7-4951-8fe1-2669d818c070\") " pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:56.598463 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.598438 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:56.675289 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.675251 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlsgp\" (UniqueName: \"kubernetes.io/projected/7ff24439-bb83-45d5-bbc5-3f08c972ca58-kube-api-access-qlsgp\") pod \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " Jul 08 16:43:56.675488 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.675331 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert\") pod \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\" (UID: \"7ff24439-bb83-45d5-bbc5-3f08c972ca58\") " Jul 08 16:43:56.677694 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.677667 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert" (OuterVolumeSpecName: "cert") pod "7ff24439-bb83-45d5-bbc5-3f08c972ca58" (UID: "7ff24439-bb83-45d5-bbc5-3f08c972ca58"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:43:56.677782 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.677696 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff24439-bb83-45d5-bbc5-3f08c972ca58-kube-api-access-qlsgp" (OuterVolumeSpecName: "kube-api-access-qlsgp") pod "7ff24439-bb83-45d5-bbc5-3f08c972ca58" (UID: "7ff24439-bb83-45d5-bbc5-3f08c972ca58"). InnerVolumeSpecName "kube-api-access-qlsgp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:43:56.716016 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.715910 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:56.776602 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.776572 2578 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ff24439-bb83-45d5-bbc5-3f08c972ca58-cert\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:43:56.776602 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.776603 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlsgp\" (UniqueName: \"kubernetes.io/projected/7ff24439-bb83-45d5-bbc5-3f08c972ca58-kube-api-access-qlsgp\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:43:56.851977 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:56.851947 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-pmx8q"] Jul 08 16:43:56.854692 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:43:56.854656 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcccff085_ade7_4951_8fe1_2669d818c070.slice/crio-4dce1a3ed6a7ba746d9b558a2cf6d84c068054d517a9fba11770e54292654319 WatchSource:0}: Error finding container 4dce1a3ed6a7ba746d9b558a2cf6d84c068054d517a9fba11770e54292654319: Status 404 returned error can't find the container with id 4dce1a3ed6a7ba746d9b558a2cf6d84c068054d517a9fba11770e54292654319 Jul 08 16:43:57.307129 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.307097 2578 generic.go:356] "Generic (PLEG): container finished" podID="7ff24439-bb83-45d5-bbc5-3f08c972ca58" containerID="146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239" exitCode=0 Jul 08 16:43:57.307605 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.307163 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" Jul 08 16:43:57.307605 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.307168 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" event={"ID":"7ff24439-bb83-45d5-bbc5-3f08c972ca58","Type":"ContainerDied","Data":"146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239"} Jul 08 16:43:57.307605 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.307312 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-c8c545f5f-4f7nw" event={"ID":"7ff24439-bb83-45d5-bbc5-3f08c972ca58","Type":"ContainerDied","Data":"31f9965ee9ef4f66ebdfa25f561be04bf419f897ad8e9244d4d67ec85ec872f6"} Jul 08 16:43:57.307605 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.307349 2578 scope.go:117] "RemoveContainer" containerID="146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239" Jul 08 16:43:57.309124 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.309009 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" event={"ID":"cccff085-ade7-4951-8fe1-2669d818c070","Type":"ContainerStarted","Data":"4ac583a348e751ac9bf22a1e25db1c899733622bae1c15bab5f0dab7ed1ca69f"} Jul 08 16:43:57.309124 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.309062 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" event={"ID":"cccff085-ade7-4951-8fe1-2669d818c070","Type":"ContainerStarted","Data":"4dce1a3ed6a7ba746d9b558a2cf6d84c068054d517a9fba11770e54292654319"} Jul 08 16:43:57.309419 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.309370 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:43:57.316121 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.316100 2578 scope.go:117] "RemoveContainer" containerID="146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239" Jul 08 16:43:57.316455 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:43:57.316429 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239\": container with ID starting with 146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239 not found: ID does not exist" containerID="146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239" Jul 08 16:43:57.316518 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.316461 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239"} err="failed to get container status \"146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239\": rpc error: code = NotFound desc = could not find container \"146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239\": container with ID starting with 146d692e900c5abd28a558b2a61b28dd5a117ca59529bde049a0245f4e761239 not found: ID does not exist" Jul 08 16:43:57.328951 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.328898 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" podStartSLOduration=1.005265373 podStartE2EDuration="1.328880946s" totalImagesPullingTime="323.615573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:43:57.327564173 +0000 UTC m=+563.054489068" watchObservedRunningTime="2026-07-08 16:43:57.328880946 +0000 UTC m=+563.055805819" Jul 08 16:43:57.343303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.343250 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-4f7nw"] Jul 08 16:43:57.347140 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:57.347113 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-c8c545f5f-4f7nw"] Jul 08 16:43:58.741603 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:43:58.741571 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ff24439-bb83-45d5-bbc5-3f08c972ca58" path="/var/lib/kubelet/pods/7ff24439-bb83-45d5-bbc5-3f08c972ca58/volumes" Jul 08 16:44:28.319894 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:28.319862 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-c8c545f5f-pmx8q" Jul 08 16:44:45.967293 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:45.967252 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-fqrm4"] Jul 08 16:44:45.967760 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:45.967742 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff24439-bb83-45d5-bbc5-3f08c972ca58" containerName="manager" Jul 08 16:44:45.967760 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:45.967760 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff24439-bb83-45d5-bbc5-3f08c972ca58" containerName="manager" Jul 08 16:44:45.967887 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:45.967871 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff24439-bb83-45d5-bbc5-3f08c972ca58" containerName="manager" Jul 08 16:44:45.972135 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:45.972111 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-fqrm4" Jul 08 16:44:45.974652 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:45.974632 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 08 16:44:45.974788 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:45.974757 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-mcq4x\"" Jul 08 16:44:45.985417 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:45.985390 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-fqrm4"] Jul 08 16:44:46.008627 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.008592 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md2b8\" (UniqueName: \"kubernetes.io/projected/97b09c76-7744-4803-8d0d-b67b22028fd2-kube-api-access-md2b8\") pod \"s3-init-fqrm4\" (UID: \"97b09c76-7744-4803-8d0d-b67b22028fd2\") " pod="kserve/s3-init-fqrm4" Jul 08 16:44:46.008627 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.008642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b09c76-7744-4803-8d0d-b67b22028fd2-model-cache\") pod \"s3-init-fqrm4\" (UID: \"97b09c76-7744-4803-8d0d-b67b22028fd2\") " pod="kserve/s3-init-fqrm4" Jul 08 16:44:46.109536 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.109494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md2b8\" (UniqueName: \"kubernetes.io/projected/97b09c76-7744-4803-8d0d-b67b22028fd2-kube-api-access-md2b8\") pod \"s3-init-fqrm4\" (UID: \"97b09c76-7744-4803-8d0d-b67b22028fd2\") " pod="kserve/s3-init-fqrm4" Jul 08 16:44:46.109715 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.109546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b09c76-7744-4803-8d0d-b67b22028fd2-model-cache\") pod \"s3-init-fqrm4\" (UID: \"97b09c76-7744-4803-8d0d-b67b22028fd2\") " pod="kserve/s3-init-fqrm4" Jul 08 16:44:46.109913 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.109898 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b09c76-7744-4803-8d0d-b67b22028fd2-model-cache\") pod \"s3-init-fqrm4\" (UID: \"97b09c76-7744-4803-8d0d-b67b22028fd2\") " pod="kserve/s3-init-fqrm4" Jul 08 16:44:46.119886 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.119847 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md2b8\" (UniqueName: \"kubernetes.io/projected/97b09c76-7744-4803-8d0d-b67b22028fd2-kube-api-access-md2b8\") pod \"s3-init-fqrm4\" (UID: \"97b09c76-7744-4803-8d0d-b67b22028fd2\") " pod="kserve/s3-init-fqrm4" Jul 08 16:44:46.292167 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.292067 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-fqrm4" Jul 08 16:44:46.433229 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.433194 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-fqrm4"] Jul 08 16:44:46.436147 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:44:46.436121 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97b09c76_7744_4803_8d0d_b67b22028fd2.slice/crio-23ad4e307d32cc805e260c616a874aa1b51b5656dab65944039fbc7aca552236 WatchSource:0}: Error finding container 23ad4e307d32cc805e260c616a874aa1b51b5656dab65944039fbc7aca552236: Status 404 returned error can't find the container with id 23ad4e307d32cc805e260c616a874aa1b51b5656dab65944039fbc7aca552236 Jul 08 16:44:46.513789 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:46.513744 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-fqrm4" event={"ID":"97b09c76-7744-4803-8d0d-b67b22028fd2","Type":"ContainerStarted","Data":"23ad4e307d32cc805e260c616a874aa1b51b5656dab65944039fbc7aca552236"} Jul 08 16:44:50.536426 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:50.536387 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-fqrm4" event={"ID":"97b09c76-7744-4803-8d0d-b67b22028fd2","Type":"ContainerStarted","Data":"208f7836823c479beb38743fef5db54a4898393a8c66894afb2b04beeda215b7"} Jul 08 16:44:52.550219 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:52.550184 2578 generic.go:356] "Generic (PLEG): container finished" podID="97b09c76-7744-4803-8d0d-b67b22028fd2" containerID="208f7836823c479beb38743fef5db54a4898393a8c66894afb2b04beeda215b7" exitCode=0 Jul 08 16:44:52.550647 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:52.550248 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-fqrm4" event={"ID":"97b09c76-7744-4803-8d0d-b67b22028fd2","Type":"ContainerDied","Data":"208f7836823c479beb38743fef5db54a4898393a8c66894afb2b04beeda215b7"} Jul 08 16:44:57.579413 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:57.579369 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-fqrm4" event={"ID":"97b09c76-7744-4803-8d0d-b67b22028fd2","Type":"ContainerStarted","Data":"2040390190cc89abb9a22cbdcaac329d60599e7fec5507bf795ac215743f4a3a"} Jul 08 16:44:57.595699 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:44:57.595644 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-fqrm4" podStartSLOduration=2.972005141 podStartE2EDuration="12.595627512s" totalImagesPullingTime="7.842166471s" totalInitContainerRuntime="1.7814559s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:44:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:44:57.594844077 +0000 UTC m=+623.321768958" watchObservedRunningTime="2026-07-08 16:44:57.595627512 +0000 UTC m=+623.322552383" Jul 08 16:46:02.852360 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:02.852314 2578 generic.go:356] "Generic (PLEG): container finished" podID="97b09c76-7744-4803-8d0d-b67b22028fd2" containerID="2040390190cc89abb9a22cbdcaac329d60599e7fec5507bf795ac215743f4a3a" exitCode=0 Jul 08 16:46:02.852769 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:02.852386 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-fqrm4" event={"ID":"97b09c76-7744-4803-8d0d-b67b22028fd2","Type":"ContainerDied","Data":"2040390190cc89abb9a22cbdcaac329d60599e7fec5507bf795ac215743f4a3a"} Jul 08 16:46:03.991745 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:03.991699 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-fqrm4" Jul 08 16:46:04.026741 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.026706 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b09c76-7744-4803-8d0d-b67b22028fd2-model-cache\") pod \"97b09c76-7744-4803-8d0d-b67b22028fd2\" (UID: \"97b09c76-7744-4803-8d0d-b67b22028fd2\") " Jul 08 16:46:04.026935 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.026753 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md2b8\" (UniqueName: \"kubernetes.io/projected/97b09c76-7744-4803-8d0d-b67b22028fd2-kube-api-access-md2b8\") pod \"97b09c76-7744-4803-8d0d-b67b22028fd2\" (UID: \"97b09c76-7744-4803-8d0d-b67b22028fd2\") " Jul 08 16:46:04.029256 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.029216 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b09c76-7744-4803-8d0d-b67b22028fd2-kube-api-access-md2b8" (OuterVolumeSpecName: "kube-api-access-md2b8") pod "97b09c76-7744-4803-8d0d-b67b22028fd2" (UID: "97b09c76-7744-4803-8d0d-b67b22028fd2"). InnerVolumeSpecName "kube-api-access-md2b8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:46:04.045878 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.045832 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97b09c76-7744-4803-8d0d-b67b22028fd2-model-cache" (OuterVolumeSpecName: "model-cache") pod "97b09c76-7744-4803-8d0d-b67b22028fd2" (UID: "97b09c76-7744-4803-8d0d-b67b22028fd2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:46:04.127445 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.127351 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97b09c76-7744-4803-8d0d-b67b22028fd2-model-cache\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:46:04.127445 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.127388 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md2b8\" (UniqueName: \"kubernetes.io/projected/97b09c76-7744-4803-8d0d-b67b22028fd2-kube-api-access-md2b8\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:46:04.862826 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.862792 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-fqrm4" Jul 08 16:46:04.863156 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.862812 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-fqrm4" event={"ID":"97b09c76-7744-4803-8d0d-b67b22028fd2","Type":"ContainerDied","Data":"23ad4e307d32cc805e260c616a874aa1b51b5656dab65944039fbc7aca552236"} Jul 08 16:46:04.863314 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:04.863177 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23ad4e307d32cc805e260c616a874aa1b51b5656dab65944039fbc7aca552236" Jul 08 16:46:17.907599 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.907564 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t"] Jul 08 16:46:17.907961 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.907912 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b09c76-7744-4803-8d0d-b67b22028fd2" containerName="s3-init" Jul 08 16:46:17.907961 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.907924 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b09c76-7744-4803-8d0d-b67b22028fd2" containerName="s3-init" Jul 08 16:46:17.907961 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.907938 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b09c76-7744-4803-8d0d-b67b22028fd2" containerName="download-hf-model" Jul 08 16:46:17.907961 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.907943 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b09c76-7744-4803-8d0d-b67b22028fd2" containerName="download-hf-model" Jul 08 16:46:17.908090 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.907992 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b09c76-7744-4803-8d0d-b67b22028fd2" containerName="s3-init" Jul 08 16:46:17.911565 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.911543 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:17.913958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.913932 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 08 16:46:17.914317 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.914287 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 08 16:46:17.914800 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.914662 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 08 16:46:17.914800 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.914689 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gs76g\"" Jul 08 16:46:17.914800 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.914715 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 08 16:46:17.914800 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.914740 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 08 16:46:17.915102 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.915016 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 08 16:46:17.926792 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:17.926763 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t"] Jul 08 16:46:18.051633 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.051597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.051802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.051672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf187438-a689-45c2-8fba-bdc213938cda-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.051802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.051692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpc2w\" (UniqueName: \"kubernetes.io/projected/cf187438-a689-45c2-8fba-bdc213938cda-kube-api-access-qpc2w\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.051802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.051715 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.051802 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.051733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf187438-a689-45c2-8fba-bdc213938cda-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.152258 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.152223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.152486 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.152328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf187438-a689-45c2-8fba-bdc213938cda-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.152486 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.152360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpc2w\" (UniqueName: \"kubernetes.io/projected/cf187438-a689-45c2-8fba-bdc213938cda-kube-api-access-qpc2w\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.152486 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.152396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.152486 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.152427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf187438-a689-45c2-8fba-bdc213938cda-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.152883 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.152858 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf187438-a689-45c2-8fba-bdc213938cda-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.153058 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.153040 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.153126 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.153110 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.155083 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.155064 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf187438-a689-45c2-8fba-bdc213938cda-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.164506 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.164438 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpc2w\" (UniqueName: \"kubernetes.io/projected/cf187438-a689-45c2-8fba-bdc213938cda-kube-api-access-qpc2w\") pod \"isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.204294 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.204245 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc"] Jul 08 16:46:18.209831 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.209799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.212483 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.212456 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 08 16:46:18.212634 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.212603 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 08 16:46:18.219571 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.219548 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc"] Jul 08 16:46:18.222985 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.222956 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:18.253292 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.253230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9mbm\" (UniqueName: \"kubernetes.io/projected/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-api-access-n9mbm\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.253466 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.253293 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe3de155-abb6-455d-99cb-99ed3a204f6d-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.253466 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.253424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.253596 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.253499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe3de155-abb6-455d-99cb-99ed3a204f6d-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.253719 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.253681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.354938 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.354882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9mbm\" (UniqueName: \"kubernetes.io/projected/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-api-access-n9mbm\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.355123 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.354945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe3de155-abb6-455d-99cb-99ed3a204f6d-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.355123 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.354995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.355123 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.355059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe3de155-abb6-455d-99cb-99ed3a204f6d-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.355123 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.355105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.355460 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.355431 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe3de155-abb6-455d-99cb-99ed3a204f6d-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.356140 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.356102 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.356247 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.356123 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.358572 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.358528 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe3de155-abb6-455d-99cb-99ed3a204f6d-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.363714 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.363685 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9mbm\" (UniqueName: \"kubernetes.io/projected/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-api-access-n9mbm\") pod \"isvc-xgboost-graph-predictor-555fc69d56-h82kc\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.383010 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.382983 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t"] Jul 08 16:46:18.385270 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:46:18.385242 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf187438_a689_45c2_8fba_bdc213938cda.slice/crio-71e52a4a309a98e8aeb27c98df28a8cf385e1f6f00ceedd3a23c431cd6312dac WatchSource:0}: Error finding container 71e52a4a309a98e8aeb27c98df28a8cf385e1f6f00ceedd3a23c431cd6312dac: Status 404 returned error can't find the container with id 71e52a4a309a98e8aeb27c98df28a8cf385e1f6f00ceedd3a23c431cd6312dac Jul 08 16:46:18.410486 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.410454 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8"] Jul 08 16:46:18.415852 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.415799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.417783 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.417761 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 08 16:46:18.417906 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.417780 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 08 16:46:18.423564 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.423535 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8"] Jul 08 16:46:18.524219 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.524189 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:18.556670 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.556635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b0a9047-2857-4191-b645-b3297003c8a7-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.556835 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.556676 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.556835 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.556727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b0a9047-2857-4191-b645-b3297003c8a7-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.556835 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.556757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.556950 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.556847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6btq\" (UniqueName: \"kubernetes.io/projected/0b0a9047-2857-4191-b645-b3297003c8a7-kube-api-access-c6btq\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.657793 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.657757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6btq\" (UniqueName: \"kubernetes.io/projected/0b0a9047-2857-4191-b645-b3297003c8a7-kube-api-access-c6btq\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.657982 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.657882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b0a9047-2857-4191-b645-b3297003c8a7-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.657982 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.657923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.658182 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.658158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b0a9047-2857-4191-b645-b3297003c8a7-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.658259 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.658237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.658436 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.658412 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b0a9047-2857-4191-b645-b3297003c8a7-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.658762 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.658740 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.659054 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.659028 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.661043 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.661019 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b0a9047-2857-4191-b645-b3297003c8a7-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.665960 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.665898 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6btq\" (UniqueName: \"kubernetes.io/projected/0b0a9047-2857-4191-b645-b3297003c8a7-kube-api-access-c6btq\") pod \"isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.687361 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.687326 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc"] Jul 08 16:46:18.687783 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:46:18.687755 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3de155_abb6_455d_99cb_99ed3a204f6d.slice/crio-9b72aafc155285c70b22650376f7584e1cccb13301d7bd08aafd3238bd103d3d WatchSource:0}: Error finding container 9b72aafc155285c70b22650376f7584e1cccb13301d7bd08aafd3238bd103d3d: Status 404 returned error can't find the container with id 9b72aafc155285c70b22650376f7584e1cccb13301d7bd08aafd3238bd103d3d Jul 08 16:46:18.741347 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.741318 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:18.891171 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.891134 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8"] Jul 08 16:46:18.895787 ip-10-0-130-254 kubenswrapper[2578]: W0708 16:46:18.895744 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0a9047_2857_4191_b645_b3297003c8a7.slice/crio-64e8021288b9c239c27f7d7697218bdd2e847831ccc458a6a94d8300afb36c92 WatchSource:0}: Error finding container 64e8021288b9c239c27f7d7697218bdd2e847831ccc458a6a94d8300afb36c92: Status 404 returned error can't find the container with id 64e8021288b9c239c27f7d7697218bdd2e847831ccc458a6a94d8300afb36c92 Jul 08 16:46:18.922351 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.922244 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" event={"ID":"fe3de155-abb6-455d-99cb-99ed3a204f6d","Type":"ContainerStarted","Data":"51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620"} Jul 08 16:46:18.923413 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.923115 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" event={"ID":"fe3de155-abb6-455d-99cb-99ed3a204f6d","Type":"ContainerStarted","Data":"9b72aafc155285c70b22650376f7584e1cccb13301d7bd08aafd3238bd103d3d"} Jul 08 16:46:18.923872 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.923846 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" event={"ID":"0b0a9047-2857-4191-b645-b3297003c8a7","Type":"ContainerStarted","Data":"64e8021288b9c239c27f7d7697218bdd2e847831ccc458a6a94d8300afb36c92"} Jul 08 16:46:18.925665 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.925640 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" event={"ID":"cf187438-a689-45c2-8fba-bdc213938cda","Type":"ContainerStarted","Data":"0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef"} Jul 08 16:46:18.925789 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:18.925702 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" event={"ID":"cf187438-a689-45c2-8fba-bdc213938cda","Type":"ContainerStarted","Data":"71e52a4a309a98e8aeb27c98df28a8cf385e1f6f00ceedd3a23c431cd6312dac"} Jul 08 16:46:19.931645 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:19.931610 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" event={"ID":"0b0a9047-2857-4191-b645-b3297003c8a7","Type":"ContainerStarted","Data":"c0cb4c42def36a6e21b8b1d3b82416397c14b922ae37d993e25fa93f129385d1"} Jul 08 16:46:22.948922 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:22.948888 2578 generic.go:356] "Generic (PLEG): container finished" podID="0b0a9047-2857-4191-b645-b3297003c8a7" containerID="c0cb4c42def36a6e21b8b1d3b82416397c14b922ae37d993e25fa93f129385d1" exitCode=0 Jul 08 16:46:22.949387 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:22.948974 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" event={"ID":"0b0a9047-2857-4191-b645-b3297003c8a7","Type":"ContainerDied","Data":"c0cb4c42def36a6e21b8b1d3b82416397c14b922ae37d993e25fa93f129385d1"} Jul 08 16:46:22.950761 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:22.950732 2578 generic.go:356] "Generic (PLEG): container finished" podID="cf187438-a689-45c2-8fba-bdc213938cda" containerID="0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef" exitCode=0 Jul 08 16:46:22.950891 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:22.950821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" event={"ID":"cf187438-a689-45c2-8fba-bdc213938cda","Type":"ContainerDied","Data":"0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef"} Jul 08 16:46:22.952559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:22.952531 2578 generic.go:356] "Generic (PLEG): container finished" podID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerID="51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620" exitCode=0 Jul 08 16:46:22.952708 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:22.952558 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" event={"ID":"fe3de155-abb6-455d-99cb-99ed3a204f6d","Type":"ContainerDied","Data":"51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620"} Jul 08 16:46:52.165050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:52.164960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" event={"ID":"0b0a9047-2857-4191-b645-b3297003c8a7","Type":"ContainerStarted","Data":"e0401eeb918e09a6bdabe74bd396a35545f79b08fe5ca997889295d283dd7fca"} Jul 08 16:46:53.176487 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:53.176439 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" event={"ID":"fe3de155-abb6-455d-99cb-99ed3a204f6d","Type":"ContainerStarted","Data":"53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5"} Jul 08 16:46:53.180584 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:53.180513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" event={"ID":"cf187438-a689-45c2-8fba-bdc213938cda","Type":"ContainerStarted","Data":"cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55"} Jul 08 16:46:55.195629 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.195537 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" event={"ID":"0b0a9047-2857-4191-b645-b3297003c8a7","Type":"ContainerStarted","Data":"95b1c32c2740e7948a83a13536ee6d3adbf46994b4d84c32decba8b89e12709b"} Jul 08 16:46:55.196084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.195757 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:55.196084 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.195802 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:46:55.197266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.197229 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:46:55.198123 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.198093 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" event={"ID":"cf187438-a689-45c2-8fba-bdc213938cda","Type":"ContainerStarted","Data":"7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099"} Jul 08 16:46:55.198265 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.198155 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:55.198265 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.198187 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:46:55.199441 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.199406 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:46:55.200528 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.200508 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" event={"ID":"fe3de155-abb6-455d-99cb-99ed3a204f6d","Type":"ContainerStarted","Data":"51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748"} Jul 08 16:46:55.200654 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.200641 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:55.221597 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.221526 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podStartSLOduration=1.726835465 podStartE2EDuration="37.221498396s" totalImagesPullingTime="31.884562971s" totalInitContainerRuntime="3.61009996s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:46:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:46:55.219445932 +0000 UTC m=+740.946370816" watchObservedRunningTime="2026-07-08 16:46:55.221498396 +0000 UTC m=+740.948423270" Jul 08 16:46:55.241847 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.241781 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podStartSLOduration=2.33566607 podStartE2EDuration="38.241756543s" totalImagesPullingTime="31.884573483s" totalInitContainerRuntime="4.02151699s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:46:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:46:55.240697078 +0000 UTC m=+740.967621954" watchObservedRunningTime="2026-07-08 16:46:55.241756543 +0000 UTC m=+740.968681415" Jul 08 16:46:55.261780 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:55.261713 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podStartSLOduration=1.4656403660000001 podStartE2EDuration="37.261688289s" totalImagesPullingTime="31.855377681s" totalInitContainerRuntime="3.940670242s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:46:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:46:55.260694918 +0000 UTC m=+740.987619792" watchObservedRunningTime="2026-07-08 16:46:55.261688289 +0000 UTC m=+740.988613163" Jul 08 16:46:56.205531 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:56.205488 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:46:56.205982 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:56.205801 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:46:56.206050 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:56.205985 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:46:56.206684 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:56.206662 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 08 16:46:57.214793 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:46:57.214748 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 08 16:47:01.209839 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:01.209808 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:47:01.210353 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:01.210218 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:47:01.210353 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:01.210332 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:47:01.210750 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:01.210728 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:47:02.219568 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:02.219538 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:47:02.220215 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:02.220184 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 08 16:47:11.211076 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:11.211022 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:47:11.211596 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:11.211022 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:47:12.220288 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:12.220218 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 08 16:47:21.210367 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:21.210327 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:47:21.210913 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:21.210687 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:47:22.220303 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:22.220230 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 08 16:47:31.211073 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:31.211030 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:47:31.211660 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:31.211030 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:47:32.220443 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:32.220398 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 08 16:47:41.210266 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:41.210226 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:47:41.210756 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:41.210678 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:47:42.220385 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:42.220344 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 08 16:47:51.210698 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:51.210657 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:47:51.211173 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:51.210657 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:47:52.220301 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:47:52.220229 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 08 16:48:01.211053 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:01.210962 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:48:01.211539 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:01.210963 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:48:02.220498 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:02.220458 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:48:11.211220 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:11.211187 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:48:11.211724 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:11.211328 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:48:48.013615 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.013580 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8"] Jul 08 16:48:48.014112 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.013887 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" containerID="cri-o://e0401eeb918e09a6bdabe74bd396a35545f79b08fe5ca997889295d283dd7fca" gracePeriod=30 Jul 08 16:48:48.014112 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.013960 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kube-rbac-proxy" containerID="cri-o://95b1c32c2740e7948a83a13536ee6d3adbf46994b4d84c32decba8b89e12709b" gracePeriod=30 Jul 08 16:48:48.086531 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.086488 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t"] Jul 08 16:48:48.086882 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.086824 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" containerID="cri-o://cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55" gracePeriod=30 Jul 08 16:48:48.087044 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.086859 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kube-rbac-proxy" containerID="cri-o://7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099" gracePeriod=30 Jul 08 16:48:48.136759 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.136721 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc"] Jul 08 16:48:48.137060 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.137035 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" containerID="cri-o://53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5" gracePeriod=30 Jul 08 16:48:48.137138 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.137078 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kube-rbac-proxy" containerID="cri-o://51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748" gracePeriod=30 Jul 08 16:48:48.736146 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.736103 2578 generic.go:356] "Generic (PLEG): container finished" podID="0b0a9047-2857-4191-b645-b3297003c8a7" containerID="95b1c32c2740e7948a83a13536ee6d3adbf46994b4d84c32decba8b89e12709b" exitCode=2 Jul 08 16:48:48.738404 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.738378 2578 generic.go:356] "Generic (PLEG): container finished" podID="cf187438-a689-45c2-8fba-bdc213938cda" containerID="7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099" exitCode=2 Jul 08 16:48:48.740750 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.740719 2578 generic.go:356] "Generic (PLEG): container finished" podID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerID="51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748" exitCode=2 Jul 08 16:48:48.741851 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.741816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" event={"ID":"0b0a9047-2857-4191-b645-b3297003c8a7","Type":"ContainerDied","Data":"95b1c32c2740e7948a83a13536ee6d3adbf46994b4d84c32decba8b89e12709b"} Jul 08 16:48:48.741966 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.741886 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" event={"ID":"cf187438-a689-45c2-8fba-bdc213938cda","Type":"ContainerDied","Data":"7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099"} Jul 08 16:48:48.741966 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:48.741914 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" event={"ID":"fe3de155-abb6-455d-99cb-99ed3a204f6d","Type":"ContainerDied","Data":"51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748"} Jul 08 16:48:51.206467 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:51.206407 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused" Jul 08 16:48:51.206467 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:51.206407 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused" Jul 08 16:48:51.210771 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:51.210736 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 08 16:48:51.210875 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:51.210736 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Jul 08 16:48:52.290616 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.290584 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:48:52.325225 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325190 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe3de155-abb6-455d-99cb-99ed3a204f6d-proxy-tls\") pod \"fe3de155-abb6-455d-99cb-99ed3a204f6d\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " Jul 08 16:48:52.325417 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325249 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9mbm\" (UniqueName: \"kubernetes.io/projected/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-api-access-n9mbm\") pod \"fe3de155-abb6-455d-99cb-99ed3a204f6d\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " Jul 08 16:48:52.325417 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325308 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe3de155-abb6-455d-99cb-99ed3a204f6d-kserve-provision-location\") pod \"fe3de155-abb6-455d-99cb-99ed3a204f6d\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " Jul 08 16:48:52.325526 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325414 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-rbac-proxy-sar-config\") pod \"fe3de155-abb6-455d-99cb-99ed3a204f6d\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " Jul 08 16:48:52.325589 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325522 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-cabundle-cert\") pod \"fe3de155-abb6-455d-99cb-99ed3a204f6d\" (UID: \"fe3de155-abb6-455d-99cb-99ed3a204f6d\") " Jul 08 16:48:52.325740 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325696 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe3de155-abb6-455d-99cb-99ed3a204f6d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "fe3de155-abb6-455d-99cb-99ed3a204f6d" (UID: "fe3de155-abb6-455d-99cb-99ed3a204f6d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:48:52.325879 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325795 2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe3de155-abb6-455d-99cb-99ed3a204f6d-kserve-provision-location\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:52.325958 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325927 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fe3de155-abb6-455d-99cb-99ed3a204f6d" (UID: "fe3de155-abb6-455d-99cb-99ed3a204f6d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:48:52.326023 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.325961 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "fe3de155-abb6-455d-99cb-99ed3a204f6d" (UID: "fe3de155-abb6-455d-99cb-99ed3a204f6d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:48:52.328658 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.328630 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-api-access-n9mbm" (OuterVolumeSpecName: "kube-api-access-n9mbm") pod "fe3de155-abb6-455d-99cb-99ed3a204f6d" (UID: "fe3de155-abb6-455d-99cb-99ed3a204f6d"). InnerVolumeSpecName "kube-api-access-n9mbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:48:52.328658 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.328647 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe3de155-abb6-455d-99cb-99ed3a204f6d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fe3de155-abb6-455d-99cb-99ed3a204f6d" (UID: "fe3de155-abb6-455d-99cb-99ed3a204f6d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:48:52.427474 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.427432 2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe3de155-abb6-455d-99cb-99ed3a204f6d-proxy-tls\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:52.427651 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.427479 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9mbm\" (UniqueName: \"kubernetes.io/projected/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-api-access-n9mbm\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:52.427651 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.427501 2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:52.427651 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.427520 2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe3de155-abb6-455d-99cb-99ed3a204f6d-cabundle-cert\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:52.761002 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.760972 2578 generic.go:356] "Generic (PLEG): container finished" podID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerID="53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5" exitCode=0 Jul 08 16:48:52.761190 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.761041 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" Jul 08 16:48:52.761190 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.761064 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" event={"ID":"fe3de155-abb6-455d-99cb-99ed3a204f6d","Type":"ContainerDied","Data":"53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5"} Jul 08 16:48:52.761190 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.761138 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" event={"ID":"fe3de155-abb6-455d-99cb-99ed3a204f6d","Type":"ContainerDied","Data":"9b72aafc155285c70b22650376f7584e1cccb13301d7bd08aafd3238bd103d3d"} Jul 08 16:48:52.761190 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.761177 2578 scope.go:117] "RemoveContainer" containerID="51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748" Jul 08 16:48:52.770158 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.770130 2578 scope.go:117] "RemoveContainer" containerID="53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5" Jul 08 16:48:52.777596 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.777568 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc"] Jul 08 16:48:52.779414 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.779390 2578 scope.go:117] "RemoveContainer" containerID="51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620" Jul 08 16:48:52.781649 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.781629 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc"] Jul 08 16:48:52.787861 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.787837 2578 scope.go:117] "RemoveContainer" containerID="51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748" Jul 08 16:48:52.788179 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:48:52.788159 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748\": container with ID starting with 51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748 not found: ID does not exist" containerID="51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748" Jul 08 16:48:52.788243 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.788190 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748"} err="failed to get container status \"51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748\": rpc error: code = NotFound desc = could not find container \"51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748\": container with ID starting with 51bc568549cb542def4adbc3678c1c74826433712aa63b651ba206074a8f4748 not found: ID does not exist" Jul 08 16:48:52.788243 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.788209 2578 scope.go:117] "RemoveContainer" containerID="53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5" Jul 08 16:48:52.788503 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:48:52.788484 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5\": container with ID starting with 53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5 not found: ID does not exist" containerID="53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5" Jul 08 16:48:52.788553 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.788510 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5"} err="failed to get container status \"53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5\": rpc error: code = NotFound desc = could not find container \"53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5\": container with ID starting with 53cd210c65968d9bc7b31610730d3b7a0e736f986329bd3c2dec623d6d254fb5 not found: ID does not exist" Jul 08 16:48:52.788553 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.788525 2578 scope.go:117] "RemoveContainer" containerID="51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620" Jul 08 16:48:52.788802 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:48:52.788773 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620\": container with ID starting with 51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620 not found: ID does not exist" containerID="51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620" Jul 08 16:48:52.788857 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:52.788809 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620"} err="failed to get container status \"51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620\": rpc error: code = NotFound desc = could not find container \"51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620\": container with ID starting with 51e9105c6767d0be8613065e9e424fee47cfa3da1edb4b5466c0d62a81f83620 not found: ID does not exist" Jul 08 16:48:53.215595 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.215541 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": context deadline exceeded" Jul 08 16:48:53.220940 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.220904 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-h82kc" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: i/o timeout" Jul 08 16:48:53.760891 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.760853 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:48:53.769251 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.769215 2578 generic.go:356] "Generic (PLEG): container finished" podID="0b0a9047-2857-4191-b645-b3297003c8a7" containerID="e0401eeb918e09a6bdabe74bd396a35545f79b08fe5ca997889295d283dd7fca" exitCode=0 Jul 08 16:48:53.769455 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.769301 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" event={"ID":"0b0a9047-2857-4191-b645-b3297003c8a7","Type":"ContainerDied","Data":"e0401eeb918e09a6bdabe74bd396a35545f79b08fe5ca997889295d283dd7fca"} Jul 08 16:48:53.771478 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.771449 2578 generic.go:356] "Generic (PLEG): container finished" podID="cf187438-a689-45c2-8fba-bdc213938cda" containerID="cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55" exitCode=0 Jul 08 16:48:53.771625 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.771523 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" Jul 08 16:48:53.771625 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.771523 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" event={"ID":"cf187438-a689-45c2-8fba-bdc213938cda","Type":"ContainerDied","Data":"cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55"} Jul 08 16:48:53.771625 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.771576 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t" event={"ID":"cf187438-a689-45c2-8fba-bdc213938cda","Type":"ContainerDied","Data":"71e52a4a309a98e8aeb27c98df28a8cf385e1f6f00ceedd3a23c431cd6312dac"} Jul 08 16:48:53.771625 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.771608 2578 scope.go:117] "RemoveContainer" containerID="7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099" Jul 08 16:48:53.782242 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.782213 2578 scope.go:117] "RemoveContainer" containerID="cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55" Jul 08 16:48:53.798592 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.798564 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:48:53.799602 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.799583 2578 scope.go:117] "RemoveContainer" containerID="0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef" Jul 08 16:48:53.808024 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.808005 2578 scope.go:117] "RemoveContainer" containerID="7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099" Jul 08 16:48:53.808363 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:48:53.808341 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099\": container with ID starting with 7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099 not found: ID does not exist" containerID="7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099" Jul 08 16:48:53.808420 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.808372 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099"} err="failed to get container status \"7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099\": rpc error: code = NotFound desc = could not find container \"7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099\": container with ID starting with 7b4850b6f2063244e297252f11258b81281ae0a761a5b246f90c396bad8ae099 not found: ID does not exist" Jul 08 16:48:53.808420 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.808393 2578 scope.go:117] "RemoveContainer" containerID="cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55" Jul 08 16:48:53.808664 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:48:53.808646 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55\": container with ID starting with cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55 not found: ID does not exist" containerID="cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55" Jul 08 16:48:53.808698 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.808671 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55"} err="failed to get container status \"cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55\": rpc error: code = NotFound desc = could not find container \"cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55\": container with ID starting with cea78091976284cb40a776133afdcb1b764cd3bff7e5359201c97328c302ba55 not found: ID does not exist" Jul 08 16:48:53.808698 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.808686 2578 scope.go:117] "RemoveContainer" containerID="0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef" Jul 08 16:48:53.808925 ip-10-0-130-254 kubenswrapper[2578]: E0708 16:48:53.808907 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef\": container with ID starting with 0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef not found: ID does not exist" containerID="0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef" Jul 08 16:48:53.808964 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.808929 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef"} err="failed to get container status \"0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef\": rpc error: code = NotFound desc = could not find container \"0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef\": container with ID starting with 0efa1571a5dbd3a7880f42a8dbf1ae3316592614d108ce97526dfd9f12a593ef not found: ID does not exist" Jul 08 16:48:53.842154 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842120 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpc2w\" (UniqueName: \"kubernetes.io/projected/cf187438-a689-45c2-8fba-bdc213938cda-kube-api-access-qpc2w\") pod \"cf187438-a689-45c2-8fba-bdc213938cda\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " Jul 08 16:48:53.842364 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842171 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-kube-rbac-proxy-sar-config\") pod \"0b0a9047-2857-4191-b645-b3297003c8a7\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " Jul 08 16:48:53.842364 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842188 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-cabundle-cert\") pod \"0b0a9047-2857-4191-b645-b3297003c8a7\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " Jul 08 16:48:53.842364 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842228 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-kube-rbac-proxy-sar-config\") pod \"cf187438-a689-45c2-8fba-bdc213938cda\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " Jul 08 16:48:53.842549 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842461 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b0a9047-2857-4191-b645-b3297003c8a7-kserve-provision-location\") pod \"0b0a9047-2857-4191-b645-b3297003c8a7\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " Jul 08 16:48:53.842549 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842525 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf187438-a689-45c2-8fba-bdc213938cda-proxy-tls\") pod \"cf187438-a689-45c2-8fba-bdc213938cda\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " Jul 08 16:48:53.842657 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842568 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6btq\" (UniqueName: \"kubernetes.io/projected/0b0a9047-2857-4191-b645-b3297003c8a7-kube-api-access-c6btq\") pod \"0b0a9047-2857-4191-b645-b3297003c8a7\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " Jul 08 16:48:53.842657 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842567 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "cf187438-a689-45c2-8fba-bdc213938cda" (UID: "cf187438-a689-45c2-8fba-bdc213938cda"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:48:53.842657 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842598 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-cabundle-cert\") pod \"cf187438-a689-45c2-8fba-bdc213938cda\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " Jul 08 16:48:53.842657 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842603 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "0b0a9047-2857-4191-b645-b3297003c8a7" (UID: "0b0a9047-2857-4191-b645-b3297003c8a7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:48:53.842657 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842612 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0b0a9047-2857-4191-b645-b3297003c8a7" (UID: "0b0a9047-2857-4191-b645-b3297003c8a7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:48:53.842657 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842644 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b0a9047-2857-4191-b645-b3297003c8a7-proxy-tls\") pod \"0b0a9047-2857-4191-b645-b3297003c8a7\" (UID: \"0b0a9047-2857-4191-b645-b3297003c8a7\") " Jul 08 16:48:53.842938 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842678 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf187438-a689-45c2-8fba-bdc213938cda-kserve-provision-location\") pod \"cf187438-a689-45c2-8fba-bdc213938cda\" (UID: \"cf187438-a689-45c2-8fba-bdc213938cda\") " Jul 08 16:48:53.842938 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.842810 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0a9047-2857-4191-b645-b3297003c8a7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0b0a9047-2857-4191-b645-b3297003c8a7" (UID: "0b0a9047-2857-4191-b645-b3297003c8a7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:48:53.843112 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.843049 2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.843112 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.843077 2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0b0a9047-2857-4191-b645-b3297003c8a7-kserve-provision-location\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.843112 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.843075 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf187438-a689-45c2-8fba-bdc213938cda-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "cf187438-a689-45c2-8fba-bdc213938cda" (UID: "cf187438-a689-45c2-8fba-bdc213938cda"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:48:53.843112 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.843085 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "cf187438-a689-45c2-8fba-bdc213938cda" (UID: "cf187438-a689-45c2-8fba-bdc213938cda"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:48:53.843112 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.843093 2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.843361 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.843131 2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0b0a9047-2857-4191-b645-b3297003c8a7-cabundle-cert\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.845221 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.845198 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0a9047-2857-4191-b645-b3297003c8a7-kube-api-access-c6btq" (OuterVolumeSpecName: "kube-api-access-c6btq") pod "0b0a9047-2857-4191-b645-b3297003c8a7" (UID: "0b0a9047-2857-4191-b645-b3297003c8a7"). InnerVolumeSpecName "kube-api-access-c6btq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:48:53.845332 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.845232 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf187438-a689-45c2-8fba-bdc213938cda-kube-api-access-qpc2w" (OuterVolumeSpecName: "kube-api-access-qpc2w") pod "cf187438-a689-45c2-8fba-bdc213938cda" (UID: "cf187438-a689-45c2-8fba-bdc213938cda"). InnerVolumeSpecName "kube-api-access-qpc2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:48:53.845401 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.845381 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b0a9047-2857-4191-b645-b3297003c8a7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0b0a9047-2857-4191-b645-b3297003c8a7" (UID: "0b0a9047-2857-4191-b645-b3297003c8a7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:48:53.845722 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.845704 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf187438-a689-45c2-8fba-bdc213938cda-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cf187438-a689-45c2-8fba-bdc213938cda" (UID: "cf187438-a689-45c2-8fba-bdc213938cda"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:48:53.944436 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.944398 2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b0a9047-2857-4191-b645-b3297003c8a7-proxy-tls\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.944436 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.944430 2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf187438-a689-45c2-8fba-bdc213938cda-kserve-provision-location\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.944436 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.944442 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpc2w\" (UniqueName: \"kubernetes.io/projected/cf187438-a689-45c2-8fba-bdc213938cda-kube-api-access-qpc2w\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.944687 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.944451 2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf187438-a689-45c2-8fba-bdc213938cda-proxy-tls\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.944687 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.944460 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6btq\" (UniqueName: \"kubernetes.io/projected/0b0a9047-2857-4191-b645-b3297003c8a7-kube-api-access-c6btq\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:53.944687 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:53.944468 2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf187438-a689-45c2-8fba-bdc213938cda-cabundle-cert\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 16:48:54.094593 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.094550 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t"] Jul 08 16:48:54.098391 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.098359 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-f669c5c7f-qm88t"] Jul 08 16:48:54.741341 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.741306 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf187438-a689-45c2-8fba-bdc213938cda" path="/var/lib/kubelet/pods/cf187438-a689-45c2-8fba-bdc213938cda/volumes" Jul 08 16:48:54.741823 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.741810 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" path="/var/lib/kubelet/pods/fe3de155-abb6-455d-99cb-99ed3a204f6d/volumes" Jul 08 16:48:54.779169 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.779128 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" event={"ID":"0b0a9047-2857-4191-b645-b3297003c8a7","Type":"ContainerDied","Data":"64e8021288b9c239c27f7d7697218bdd2e847831ccc458a6a94d8300afb36c92"} Jul 08 16:48:54.779674 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.779211 2578 scope.go:117] "RemoveContainer" containerID="95b1c32c2740e7948a83a13536ee6d3adbf46994b4d84c32decba8b89e12709b" Jul 08 16:48:54.779674 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.779143 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8" Jul 08 16:48:54.787789 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.787732 2578 scope.go:117] "RemoveContainer" containerID="e0401eeb918e09a6bdabe74bd396a35545f79b08fe5ca997889295d283dd7fca" Jul 08 16:48:54.795966 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.795935 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8"] Jul 08 16:48:54.796608 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.796591 2578 scope.go:117] "RemoveContainer" containerID="c0cb4c42def36a6e21b8b1d3b82416397c14b922ae37d993e25fa93f129385d1" Jul 08 16:48:54.799496 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:54.799475 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7dd9885465-zlgg8"] Jul 08 16:48:56.741559 ip-10-0-130-254 kubenswrapper[2578]: I0708 16:48:56.741515 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" path="/var/lib/kubelet/pods/0b0a9047-2857-4191-b645-b3297003c8a7/volumes" Jul 08 17:25:48.960689 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.960653 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j4q6q/must-gather-tb767"] Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.960999 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="storage-initializer" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961011 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="storage-initializer" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961020 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961025 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961033 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961038 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961045 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="storage-initializer" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961050 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="storage-initializer" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961065 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="storage-initializer" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961070 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="storage-initializer" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961077 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961083 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961089 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961094 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961103 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961109 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961118 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961123 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961184 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961192 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b0a9047-2857-4191-b645-b3297003c8a7" containerName="kserve-container" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961200 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961206 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kube-rbac-proxy" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961212 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe3de155-abb6-455d-99cb-99ed3a204f6d" containerName="kserve-container" Jul 08 17:25:48.963291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.961218 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf187438-a689-45c2-8fba-bdc213938cda" containerName="kserve-container" Jul 08 17:25:48.964325 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.964306 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:25:48.966195 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.966173 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j4q6q\"/\"kube-root-ca.crt\"" Jul 08 17:25:48.966938 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.966916 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j4q6q\"/\"openshift-service-ca.crt\"" Jul 08 17:25:48.967037 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.966916 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j4q6q\"/\"default-dockercfg-x6chh\"" Jul 08 17:25:48.975590 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:48.975562 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j4q6q/must-gather-tb767"] Jul 08 17:25:49.076907 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.076863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-must-gather-output\") pod \"must-gather-tb767\" (UID: \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\") " pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:25:49.077160 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.077003 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw6hx\" (UniqueName: \"kubernetes.io/projected/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-kube-api-access-lw6hx\") pod \"must-gather-tb767\" (UID: \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\") " pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:25:49.178021 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.177984 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw6hx\" (UniqueName: \"kubernetes.io/projected/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-kube-api-access-lw6hx\") pod \"must-gather-tb767\" (UID: \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\") " pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:25:49.178210 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.178074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-must-gather-output\") pod \"must-gather-tb767\" (UID: \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\") " pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:25:49.178469 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.178451 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-must-gather-output\") pod \"must-gather-tb767\" (UID: \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\") " pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:25:49.186591 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.186560 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw6hx\" (UniqueName: \"kubernetes.io/projected/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-kube-api-access-lw6hx\") pod \"must-gather-tb767\" (UID: \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\") " pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:25:49.285626 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.285541 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:25:49.425364 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.425314 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j4q6q/must-gather-tb767"] Jul 08 17:25:49.428295 ip-10-0-130-254 kubenswrapper[2578]: W0708 17:25:49.428248 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1606d6e3_6481_4e4b_b5d3_f66a746abcc1.slice/crio-426b943d019be7e8d2c498e63c5ab461c09c4998dda19ad2b0fc27b77e395be4 WatchSource:0}: Error finding container 426b943d019be7e8d2c498e63c5ab461c09c4998dda19ad2b0fc27b77e395be4: Status 404 returned error can't find the container with id 426b943d019be7e8d2c498e63c5ab461c09c4998dda19ad2b0fc27b77e395be4 Jul 08 17:25:49.430617 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.430600 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:25:49.964140 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:49.964100 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j4q6q/must-gather-tb767" event={"ID":"1606d6e3-6481-4e4b-b5d3-f66a746abcc1","Type":"ContainerStarted","Data":"426b943d019be7e8d2c498e63c5ab461c09c4998dda19ad2b0fc27b77e395be4"} Jul 08 17:25:55.998469 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:55.998427 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j4q6q/must-gather-tb767" event={"ID":"1606d6e3-6481-4e4b-b5d3-f66a746abcc1","Type":"ContainerStarted","Data":"8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1"} Jul 08 17:25:55.998891 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:55.998478 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j4q6q/must-gather-tb767" event={"ID":"1606d6e3-6481-4e4b-b5d3-f66a746abcc1","Type":"ContainerStarted","Data":"a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29"} Jul 08 17:25:56.014697 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:25:56.014636 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j4q6q/must-gather-tb767" podStartSLOduration=1.911905846 podStartE2EDuration="8.014619844s" totalImagesPullingTime="6.102713998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:25:56.013647422 +0000 UTC m=+3081.740572298" watchObservedRunningTime="2026-07-08 17:25:56.014619844 +0000 UTC m=+3081.741544716" Jul 08 17:26:19.109596 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:19.109485 2578 generic.go:356] "Generic (PLEG): container finished" podID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerID="a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29" exitCode=0 Jul 08 17:26:19.109596 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:19.109562 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j4q6q/must-gather-tb767" event={"ID":"1606d6e3-6481-4e4b-b5d3-f66a746abcc1","Type":"ContainerDied","Data":"a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29"} Jul 08 17:26:19.110045 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:19.109945 2578 scope.go:117] "RemoveContainer" containerID="a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29" Jul 08 17:26:19.779380 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:19.779351 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j4q6q_must-gather-tb767_1606d6e3-6481-4e4b-b5d3-f66a746abcc1/gather/0.log" Jul 08 17:26:23.670174 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:23.670145 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zbrcz_1cdda881-c366-4e95-ab7e-6491a2e4c00b/global-pull-secret-syncer/0.log" Jul 08 17:26:23.789692 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:23.789662 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s6xcp_864a8a9f-7cae-43e4-9736-2174d574a7e8/konnectivity-agent/0.log" Jul 08 17:26:23.839167 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:23.839130 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-254.ec2.internal_1b906b0da08878c33265f8e1596ed7a5/haproxy/0.log" Jul 08 17:26:25.250766 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.250722 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-j4q6q/must-gather-tb767"] Jul 08 17:26:25.251357 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.251032 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-j4q6q/must-gather-tb767" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerName="copy" containerID="cri-o://8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1" gracePeriod=2 Jul 08 17:26:25.255307 ip-10-0-130-254 kubenswrapper[2578]: E0708 17:26:25.255111 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-tb767\" is forbidden: User \"system:node:ip-10-0-130-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j4q6q\": no relationship found between node 'ip-10-0-130-254.ec2.internal' and this object" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:26:25.257918 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.257889 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-j4q6q/must-gather-tb767"] Jul 08 17:26:25.484708 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.484683 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j4q6q_must-gather-tb767_1606d6e3-6481-4e4b-b5d3-f66a746abcc1/copy/0.log" Jul 08 17:26:25.487340 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.487318 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:26:25.489092 ip-10-0-130-254 kubenswrapper[2578]: E0708 17:26:25.489068 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-tb767\" is forbidden: User \"system:node:ip-10-0-130-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j4q6q\": no relationship found between node 'ip-10-0-130-254.ec2.internal' and this object" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:26:25.609875 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.609777 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw6hx\" (UniqueName: \"kubernetes.io/projected/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-kube-api-access-lw6hx\") pod \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\" (UID: \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\") " Jul 08 17:26:25.609875 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.609835 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-must-gather-output\") pod \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\" (UID: \"1606d6e3-6481-4e4b-b5d3-f66a746abcc1\") " Jul 08 17:26:25.611474 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.611444 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "1606d6e3-6481-4e4b-b5d3-f66a746abcc1" (UID: "1606d6e3-6481-4e4b-b5d3-f66a746abcc1"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:26:25.612192 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.612165 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-kube-api-access-lw6hx" (OuterVolumeSpecName: "kube-api-access-lw6hx") pod "1606d6e3-6481-4e4b-b5d3-f66a746abcc1" (UID: "1606d6e3-6481-4e4b-b5d3-f66a746abcc1"). InnerVolumeSpecName "kube-api-access-lw6hx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:26:25.710811 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.710775 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw6hx\" (UniqueName: \"kubernetes.io/projected/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-kube-api-access-lw6hx\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 17:26:25.710811 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:25.710807 2578 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1606d6e3-6481-4e4b-b5d3-f66a746abcc1-must-gather-output\") on node \"ip-10-0-130-254.ec2.internal\" DevicePath \"\"" Jul 08 17:26:26.140259 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.140230 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j4q6q_must-gather-tb767_1606d6e3-6481-4e4b-b5d3-f66a746abcc1/copy/0.log" Jul 08 17:26:26.140637 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.140615 2578 generic.go:356] "Generic (PLEG): container finished" podID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerID="8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1" exitCode=143 Jul 08 17:26:26.140715 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.140668 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:26:26.140775 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.140726 2578 scope.go:117] "RemoveContainer" containerID="8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1" Jul 08 17:26:26.142708 ip-10-0-130-254 kubenswrapper[2578]: E0708 17:26:26.142674 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-tb767\" is forbidden: User \"system:node:ip-10-0-130-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j4q6q\": no relationship found between node 'ip-10-0-130-254.ec2.internal' and this object" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:26:26.149088 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.149069 2578 scope.go:117] "RemoveContainer" containerID="a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29" Jul 08 17:26:26.150735 ip-10-0-130-254 kubenswrapper[2578]: E0708 17:26:26.150711 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-tb767\" is forbidden: User \"system:node:ip-10-0-130-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j4q6q\": no relationship found between node 'ip-10-0-130-254.ec2.internal' and this object" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" pod="openshift-must-gather-j4q6q/must-gather-tb767" Jul 08 17:26:26.161554 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.161533 2578 scope.go:117] "RemoveContainer" containerID="8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1" Jul 08 17:26:26.161864 ip-10-0-130-254 kubenswrapper[2578]: E0708 17:26:26.161844 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1\": container with ID starting with 8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1 not found: ID does not exist" containerID="8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1" Jul 08 17:26:26.161908 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.161877 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1"} err="failed to get container status \"8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1\": rpc error: code = NotFound desc = could not find container \"8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1\": container with ID starting with 8d23d123e9a102d2408c0741cbd9e04caedc77b04521fe1fd09cc815864bf6e1 not found: ID does not exist" Jul 08 17:26:26.161908 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.161896 2578 scope.go:117] "RemoveContainer" containerID="a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29" Jul 08 17:26:26.162165 ip-10-0-130-254 kubenswrapper[2578]: E0708 17:26:26.162149 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29\": container with ID starting with a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29 not found: ID does not exist" containerID="a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29" Jul 08 17:26:26.162213 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.162171 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29"} err="failed to get container status \"a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29\": rpc error: code = NotFound desc = could not find container \"a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29\": container with ID starting with a3668ef646fbeccc85eca84e6f6f37b24c734aa41a47d5464dc5fcaf89951d29 not found: ID does not exist" Jul 08 17:26:26.741920 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:26.741888 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" path="/var/lib/kubelet/pods/1606d6e3-6481-4e4b-b5d3-f66a746abcc1/volumes" Jul 08 17:26:27.708199 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:27.708113 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8842f9e8-41d0-474d-8522-99c3a0825ccf/alertmanager/0.log" Jul 08 17:26:27.766884 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:27.766854 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8842f9e8-41d0-474d-8522-99c3a0825ccf/config-reloader/0.log" Jul 08 17:26:27.821837 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:27.821810 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8842f9e8-41d0-474d-8522-99c3a0825ccf/kube-rbac-proxy-web/0.log" Jul 08 17:26:27.867019 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:27.866994 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8842f9e8-41d0-474d-8522-99c3a0825ccf/kube-rbac-proxy/0.log" Jul 08 17:26:27.914954 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:27.914929 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8842f9e8-41d0-474d-8522-99c3a0825ccf/kube-rbac-proxy-metric/0.log" Jul 08 17:26:27.960927 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:27.960846 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8842f9e8-41d0-474d-8522-99c3a0825ccf/prom-label-proxy/0.log" Jul 08 17:26:28.004954 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.004927 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8842f9e8-41d0-474d-8522-99c3a0825ccf/init-config-reloader/0.log" Jul 08 17:26:28.119213 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.119182 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-w5xb8_3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b/kube-state-metrics/0.log" Jul 08 17:26:28.163163 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.163136 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-w5xb8_3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b/kube-rbac-proxy-main/0.log" Jul 08 17:26:28.211991 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.211915 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-w5xb8_3cb99dbf-145a-46c9-b75a-a6f0ecf1bf4b/kube-rbac-proxy-self/0.log" Jul 08 17:26:28.265016 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.264991 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b9b86cb7d-ncgl2_926d189c-09be-49bb-8442-95ae960ee0b5/metrics-server/0.log" Jul 08 17:26:28.311048 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.311013 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-gwtkp_4c0269a7-e043-41c0-a728-62a6da22a6b2/monitoring-plugin/0.log" Jul 08 17:26:28.366600 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.366562 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gm8gv_e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac/node-exporter/0.log" Jul 08 17:26:28.419037 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.419013 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gm8gv_e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac/kube-rbac-proxy/0.log" Jul 08 17:26:28.465932 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.465858 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gm8gv_e93a307e-1fde-466a-9f5e-ff4dc0f4e1ac/init-textfile/0.log" Jul 08 17:26:28.687197 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.687168 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kb62h_67cb91da-218f-41a7-bcba-cd2eb171d744/kube-rbac-proxy-main/0.log" Jul 08 17:26:28.714721 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.714696 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kb62h_67cb91da-218f-41a7-bcba-cd2eb171d744/kube-rbac-proxy-self/0.log" Jul 08 17:26:28.742440 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.742358 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kb62h_67cb91da-218f-41a7-bcba-cd2eb171d744/openshift-state-metrics/0.log" Jul 08 17:26:28.791909 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.791884 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cc2b1371-625d-443a-b632-aa0d633db569/prometheus/0.log" Jul 08 17:26:28.813497 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.813462 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cc2b1371-625d-443a-b632-aa0d633db569/config-reloader/0.log" Jul 08 17:26:28.838618 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.838570 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cc2b1371-625d-443a-b632-aa0d633db569/thanos-sidecar/0.log" Jul 08 17:26:28.862926 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.862899 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cc2b1371-625d-443a-b632-aa0d633db569/kube-rbac-proxy-web/0.log" Jul 08 17:26:28.888888 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.888845 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cc2b1371-625d-443a-b632-aa0d633db569/kube-rbac-proxy/0.log" Jul 08 17:26:28.913915 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.913889 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cc2b1371-625d-443a-b632-aa0d633db569/kube-rbac-proxy-thanos/0.log" Jul 08 17:26:28.937614 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.937585 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cc2b1371-625d-443a-b632-aa0d633db569/init-config-reloader/0.log" Jul 08 17:26:28.976135 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.976100 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-896h4_46e0a83b-036d-43d8-b6d2-ac54137630cb/prometheus-operator/0.log" Jul 08 17:26:28.998145 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:28.998059 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-896h4_46e0a83b-036d-43d8-b6d2-ac54137630cb/kube-rbac-proxy/0.log" Jul 08 17:26:29.025887 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.025858 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-tkqm7_10450850-4160-4380-b409-6ed4722dfd45/prometheus-operator-admission-webhook/0.log" Jul 08 17:26:29.059845 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.059812 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-ccb8dd978-f4xv9_ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5/telemeter-client/0.log" Jul 08 17:26:29.086347 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.086319 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-ccb8dd978-f4xv9_ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5/reload/0.log" Jul 08 17:26:29.112684 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.112656 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-ccb8dd978-f4xv9_ccb24d29-ac3e-4dbe-b19f-3d04a7a71af5/kube-rbac-proxy/0.log" Jul 08 17:26:29.147398 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.147369 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f54c9d7f-wkggn_1b05f903-b7be-484e-833f-6fe034fd5f90/thanos-query/0.log" Jul 08 17:26:29.170648 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.170617 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f54c9d7f-wkggn_1b05f903-b7be-484e-833f-6fe034fd5f90/kube-rbac-proxy-web/0.log" Jul 08 17:26:29.195134 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.195109 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f54c9d7f-wkggn_1b05f903-b7be-484e-833f-6fe034fd5f90/kube-rbac-proxy/0.log" Jul 08 17:26:29.223655 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.223626 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f54c9d7f-wkggn_1b05f903-b7be-484e-833f-6fe034fd5f90/prom-label-proxy/0.log" Jul 08 17:26:29.248581 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.248499 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f54c9d7f-wkggn_1b05f903-b7be-484e-833f-6fe034fd5f90/kube-rbac-proxy-rules/0.log" Jul 08 17:26:29.272936 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:29.272907 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68f54c9d7f-wkggn_1b05f903-b7be-484e-833f-6fe034fd5f90/kube-rbac-proxy-metrics/0.log" Jul 08 17:26:31.556080 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.556048 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz"] Jul 08 17:26:31.556557 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.556400 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerName="copy" Jul 08 17:26:31.556557 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.556413 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerName="copy" Jul 08 17:26:31.556557 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.556440 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerName="gather" Jul 08 17:26:31.556557 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.556446 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerName="gather" Jul 08 17:26:31.556557 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.556499 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerName="gather" Jul 08 17:26:31.556557 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.556507 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1606d6e3-6481-4e4b-b5d3-f66a746abcc1" containerName="copy" Jul 08 17:26:31.562056 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.562030 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.564382 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.564224 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9zkvj\"/\"kube-root-ca.crt\"" Jul 08 17:26:31.564382 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.564261 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9zkvj\"/\"openshift-service-ca.crt\"" Jul 08 17:26:31.564861 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.564839 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9zkvj\"/\"default-dockercfg-blrjz\"" Jul 08 17:26:31.567173 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.567151 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz"] Jul 08 17:26:31.663551 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.663514 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-lib-modules\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.663766 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.663568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-sys\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.663766 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.663611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-podres\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.663766 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.663649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdz5l\" (UniqueName: \"kubernetes.io/projected/f069d973-5e31-425b-a2bb-60707b1521a2-kube-api-access-cdz5l\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.663766 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.663698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-proc\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764182 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdz5l\" (UniqueName: \"kubernetes.io/projected/f069d973-5e31-425b-a2bb-60707b1521a2-kube-api-access-cdz5l\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764406 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764200 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-proc\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764406 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764249 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-lib-modules\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764406 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-sys\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764406 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764378 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-proc\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764406 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-podres\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764610 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764466 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-sys\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764610 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-podres\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.764610 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.764520 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f069d973-5e31-425b-a2bb-60707b1521a2-lib-modules\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.773871 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.773846 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdz5l\" (UniqueName: \"kubernetes.io/projected/f069d973-5e31-425b-a2bb-60707b1521a2-kube-api-access-cdz5l\") pod \"perf-node-gather-daemonset-nkdzz\" (UID: \"f069d973-5e31-425b-a2bb-60707b1521a2\") " pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:31.873298 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:31.873183 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:32.015749 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:32.015716 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz"] Jul 08 17:26:32.018946 ip-10-0-130-254 kubenswrapper[2578]: W0708 17:26:32.018902 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf069d973_5e31_425b_a2bb_60707b1521a2.slice/crio-1f00f5a67652165b1b24ac91fa0882d006eac0153ca6269769f02ef585b4dae0 WatchSource:0}: Error finding container 1f00f5a67652165b1b24ac91fa0882d006eac0153ca6269769f02ef585b4dae0: Status 404 returned error can't find the container with id 1f00f5a67652165b1b24ac91fa0882d006eac0153ca6269769f02ef585b4dae0 Jul 08 17:26:32.167742 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:32.167702 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" event={"ID":"f069d973-5e31-425b-a2bb-60707b1521a2","Type":"ContainerStarted","Data":"26c5af3153d87a4f1c15d54ca90d358a4fdee27c1742b59d35dd7dc70fb15126"} Jul 08 17:26:32.167936 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:32.167761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" event={"ID":"f069d973-5e31-425b-a2bb-60707b1521a2","Type":"ContainerStarted","Data":"1f00f5a67652165b1b24ac91fa0882d006eac0153ca6269769f02ef585b4dae0"} Jul 08 17:26:32.167936 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:32.167785 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:32.184903 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:32.184854 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" podStartSLOduration=1.184834946 podStartE2EDuration="1.184834946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:26:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:26:32.183849401 +0000 UTC m=+3117.910774283" watchObservedRunningTime="2026-07-08 17:26:32.184834946 +0000 UTC m=+3117.911759796" Jul 08 17:26:33.157669 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:33.157641 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vqswf_c51f0313-ba74-44f2-9ea8-068e01c1cf3c/dns/0.log" Jul 08 17:26:33.181085 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:33.181048 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vqswf_c51f0313-ba74-44f2-9ea8-068e01c1cf3c/kube-rbac-proxy/0.log" Jul 08 17:26:33.234249 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:33.234220 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pqjtx_ce6a0a0e-0dd4-42f0-ad7f-3d044cf5d35a/dns-node-resolver/0.log" Jul 08 17:26:33.759291 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:33.759252 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-22m2d_c58ae9ea-b1a7-4785-9987-ec3ed4f6efcc/node-ca/0.log" Jul 08 17:26:34.947855 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:34.947825 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxg57_30873582-3370-4d20-a0e9-f9bb1dd67dcd/serve-healthcheck-canary/0.log" Jul 08 17:26:35.408407 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:35.408377 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-726rr_97471e01-6997-4859-9309-8e8c09490e73/kube-rbac-proxy/0.log" Jul 08 17:26:35.431543 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:35.431512 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-726rr_97471e01-6997-4859-9309-8e8c09490e73/exporter/0.log" Jul 08 17:26:35.455045 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:35.455004 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-726rr_97471e01-6997-4859-9309-8e8c09490e73/extractor/0.log" Jul 08 17:26:37.602894 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:37.602867 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-c8c545f5f-pmx8q_cccff085-ade7-4951-8fe1-2669d818c070/manager/0.log" Jul 08 17:26:37.798733 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:37.798702 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-fqrm4_97b09c76-7744-4803-8d0d-b67b22028fd2/s3-init/0.log" Jul 08 17:26:37.817559 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:37.817530 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-fqrm4_97b09c76-7744-4803-8d0d-b67b22028fd2/download-hf-model/0.log" Jul 08 17:26:38.181959 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:38.181934 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9zkvj/perf-node-gather-daemonset-nkdzz" Jul 08 17:26:43.578725 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:43.578692 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5bnw6_0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61/kube-multus-additional-cni-plugins/0.log" Jul 08 17:26:43.602502 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:43.602480 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5bnw6_0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61/egress-router-binary-copy/0.log" Jul 08 17:26:43.626391 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:43.626360 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5bnw6_0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61/cni-plugins/0.log" Jul 08 17:26:43.650518 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:43.650492 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5bnw6_0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61/bond-cni-plugin/0.log" Jul 08 17:26:43.676801 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:43.676775 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5bnw6_0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61/routeoverride-cni/0.log" Jul 08 17:26:43.702022 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:43.701991 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5bnw6_0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61/whereabouts-cni-bincopy/0.log" Jul 08 17:26:43.726531 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:43.726500 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5bnw6_0cc854cc-e69e-4cbb-ac2f-37f52a0d8d61/whereabouts-cni/0.log" Jul 08 17:26:44.117710 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:44.117681 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lnq7s_e10fb928-0f80-4c9e-a510-297440e94a5f/kube-multus/0.log" Jul 08 17:26:44.234042 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:44.234008 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9gh26_7109969a-f717-4285-b47f-261f6b3353a2/network-metrics-daemon/0.log" Jul 08 17:26:44.269529 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:44.269500 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9gh26_7109969a-f717-4285-b47f-261f6b3353a2/kube-rbac-proxy/0.log" Jul 08 17:26:45.550433 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:45.550393 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcqgq_823540fd-8449-4516-ab31-489f22b48084/ovn-controller/0.log" Jul 08 17:26:45.570836 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:45.570812 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcqgq_823540fd-8449-4516-ab31-489f22b48084/ovn-acl-logging/0.log" Jul 08 17:26:45.596633 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:45.596602 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcqgq_823540fd-8449-4516-ab31-489f22b48084/kube-rbac-proxy-node/0.log" Jul 08 17:26:45.624467 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:45.624422 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcqgq_823540fd-8449-4516-ab31-489f22b48084/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 17:26:45.649289 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:45.649244 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcqgq_823540fd-8449-4516-ab31-489f22b48084/northd/0.log" Jul 08 17:26:45.672630 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:45.672599 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcqgq_823540fd-8449-4516-ab31-489f22b48084/nbdb/0.log" Jul 08 17:26:45.697576 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:45.697528 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcqgq_823540fd-8449-4516-ab31-489f22b48084/sbdb/0.log" Jul 08 17:26:45.742203 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:45.742169 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xcqgq_823540fd-8449-4516-ab31-489f22b48084/ovnkube-controller/0.log" Jul 08 17:26:46.829969 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:46.829936 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-knbpm_2812fd9f-259d-44ae-91fa-d839fe4b94b9/network-check-target-container/0.log" Jul 08 17:26:47.926246 ip-10-0-130-254 kubenswrapper[2578]: I0708 17:26:47.926221 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-gwjxc_16660cb9-13e7-489c-bad0-08851d9f10b3/iptables-alerter/0.log"