Jul 06 19:24:18.050905 ip-10-0-143-100 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 19:24:18.050916 ip-10-0-143-100 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 19:24:18.050926 ip-10-0-143-100 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 19:24:18.051245 ip-10-0-143-100 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 19:24:28.292157 ip-10-0-143-100 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 19:24:28.292180 ip-10-0-143-100 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a576f0371460498c8661109d46d1dbae -- Jul 06 19:26:08.484539 ip-10-0-143-100 systemd[1]: Starting Kubernetes Kubelet... Jul 06 19:26:08.966364 ip-10-0-143-100 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:26:08.966364 ip-10-0-143-100 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 19:26:08.966364 ip-10-0-143-100 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:26:08.966364 ip-10-0-143-100 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 19:26:08.966364 ip-10-0-143-100 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:26:08.966364 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.960409 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966940 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966954 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966958 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966961 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966964 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966966 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966969 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966973 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966976 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966979 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966981 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966984 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966986 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966989 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966992 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966994 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966997 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.966999 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:26:08.982328 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967009 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967012 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967015 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967019 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967023 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967026 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967030 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967034 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967037 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967040 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967043 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967046 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967049 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967051 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967054 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967057 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967059 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967062 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967064 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967067 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:26:08.997190 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967069 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967072 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967075 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967077 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967080 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967083 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967086 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967089 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967093 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967095 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967098 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967101 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967104 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967106 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967109 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967112 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967115 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967117 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967121 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967127 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:26:08.999116 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967130 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967133 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967137 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967139 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967143 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967146 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967149 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967152 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967154 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967157 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967159 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967162 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967164 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967167 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967170 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967172 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967174 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967177 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967179 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967182 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:26:09.001444 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967184 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967187 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967189 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967192 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967194 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967197 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967199 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967202 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967204 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967223 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967226 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967230 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967234 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967237 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967240 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967243 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967246 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967249 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967251 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:26:09.016643 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967254 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.967256 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969755 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969762 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969766 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969770 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969773 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969777 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969780 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969783 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969786 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969789 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969791 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969794 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969796 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969799 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969802 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969804 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969807 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969810 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:26:09.019086 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969814 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969817 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969820 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969823 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969825 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969828 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969831 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969833 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969835 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969838 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969841 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969843 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969846 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969848 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969851 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969855 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969858 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969861 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969864 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969867 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:26:09.020933 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969870 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969872 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969874 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969877 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969880 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969883 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969885 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969888 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969890 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969893 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969895 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969898 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969903 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969907 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969909 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969912 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969915 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969918 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969921 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969924 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:26:09.037013 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969926 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969929 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969931 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969934 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969937 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969940 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969942 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969945 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969947 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969950 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969952 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969955 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969957 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969962 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969965 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969967 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969970 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969972 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969975 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969977 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:26:09.039069 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969980 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969983 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969985 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969988 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969992 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969995 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.969997 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970000 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970002 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970005 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970008 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970011 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970013 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970016 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970018 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970021 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970023 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970026 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970028 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:26:09.041827 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970031 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:26:09.041828 ip-10-0-143-100 systemd[1]: Started Kubernetes Kubelet. Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970033 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970447 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970453 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970456 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970459 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970462 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970465 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970467 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970470 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970472 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970477 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970480 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970482 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970484 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970487 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970490 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970494 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970497 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970499 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:26:09.056969 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970502 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970504 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970507 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970509 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970512 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970514 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970517 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970521 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970525 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970527 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970531 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970533 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970536 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970538 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970541 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970543 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970546 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970548 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970551 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970554 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:26:09.059893 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970557 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970559 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970562 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970567 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970570 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970573 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970575 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970578 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970580 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970585 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970588 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970590 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970593 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970595 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970598 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970601 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970603 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970606 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970609 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970611 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:26:09.073804 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970613 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970616 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970618 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970621 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970623 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970626 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970628 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970630 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970633 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970636 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970638 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970641 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970644 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970647 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970649 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970653 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970655 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970658 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970660 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:26:09.076777 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970663 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970665 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970669 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970672 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970675 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970678 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970680 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970683 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970685 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970688 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970691 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970693 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970696 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970698 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970701 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970703 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970706 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970708 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970710 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:26:09.295778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970713 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970717 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.970720 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970790 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970797 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970804 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970808 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970813 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970816 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970822 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970830 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970833 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970837 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970840 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970843 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970847 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970850 2565 flags.go:64] FLAG: --cgroup-root="" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970853 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970856 2565 flags.go:64] FLAG: --client-ca-file="" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970859 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970862 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970867 2565 flags.go:64] FLAG: --cluster-domain="" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970870 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 19:26:09.297778 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970872 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970875 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970878 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970882 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970886 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970889 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970892 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970895 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970897 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970900 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970903 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970906 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970910 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970913 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970916 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970919 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970922 2565 flags.go:64] FLAG: --enable-server="true" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970925 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970931 2565 flags.go:64] FLAG: --event-burst="100" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970933 2565 flags.go:64] FLAG: --event-qps="50" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970936 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970939 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970942 2565 flags.go:64] FLAG: --eviction-hard="" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970946 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 19:26:09.309295 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970950 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970953 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970956 2565 flags.go:64] FLAG: --eviction-soft="" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970959 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970962 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970965 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970968 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970971 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970974 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970977 2565 flags.go:64] FLAG: --feature-gates="" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970981 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970984 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970987 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970990 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970993 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970996 2565 flags.go:64] FLAG: --help="false" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.970998 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971002 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971004 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971007 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971011 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971014 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971017 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 19:26:09.310095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971020 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971023 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971025 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971030 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971033 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971036 2565 flags.go:64] FLAG: --kube-reserved="" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971038 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971041 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971044 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971048 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971051 2565 flags.go:64] FLAG: --lock-file="" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971054 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971056 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971059 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971065 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971068 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971070 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971073 2565 flags.go:64] FLAG: --logging-format="text" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971076 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971079 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971082 2565 flags.go:64] FLAG: --manifest-url="" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971085 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971089 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971092 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971096 2565 flags.go:64] FLAG: --max-pods="110" Jul 06 19:26:09.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971099 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971102 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971105 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971108 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971111 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971113 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971116 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971124 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971127 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971130 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971134 2565 flags.go:64] FLAG: --pod-cidr="" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971138 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971143 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971147 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971150 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971152 2565 flags.go:64] FLAG: --port="10250" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971156 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971159 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ed0c866ffe566bd3" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971162 2565 flags.go:64] FLAG: --qos-reserved="" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971165 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971168 2565 flags.go:64] FLAG: --register-node="true" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971171 2565 flags.go:64] FLAG: --register-with-taints="" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971175 2565 flags.go:64] FLAG: --registry-burst="10" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971178 2565 flags.go:64] FLAG: --registry-qps="5" Jul 06 19:26:09.312005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971181 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971184 2565 flags.go:64] FLAG: --reserved-memory="" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971187 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971190 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971193 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971196 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971199 2565 flags.go:64] FLAG: --runonce="false" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971202 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971205 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971222 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971226 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971229 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971232 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971235 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971238 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971241 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971244 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971247 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971251 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971254 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971258 2565 flags.go:64] FLAG: --system-cgroups="" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971261 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971266 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971269 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971272 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 19:26:09.312816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971277 2565 flags.go:64] FLAG: --tls-min-version="" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971280 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971283 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971286 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971289 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971292 2565 flags.go:64] FLAG: --v="2" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971296 2565 flags.go:64] FLAG: --version="false" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971300 2565 flags.go:64] FLAG: --vmodule="" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971304 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.971307 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971430 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971434 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971438 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971441 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971443 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971447 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971450 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971453 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971455 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971458 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971460 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971463 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:26:09.313834 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971466 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971468 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971471 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971475 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971477 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971480 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971483 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971485 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971488 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971491 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971494 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971497 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971500 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971502 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971505 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971507 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971510 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971512 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971515 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:26:09.314496 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971518 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971520 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971523 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971526 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971529 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971531 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971534 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971536 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971539 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971542 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971544 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971547 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971549 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971552 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971554 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971557 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971561 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971563 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971566 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971568 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:26:09.315357 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971571 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971574 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971576 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971580 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971582 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971585 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971587 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971590 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971592 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971595 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971598 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971600 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971602 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971606 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971610 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971612 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971615 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971618 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971621 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:26:09.315946 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971623 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971626 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971630 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971633 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971636 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971639 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971641 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971645 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971648 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971653 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971656 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971659 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971661 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971664 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971667 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971669 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971673 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971675 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971679 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:26:09.316573 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971681 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971684 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971686 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971689 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971691 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971694 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971696 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971699 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971701 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.971704 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.972510 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.980276 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.980293 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.980308 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.980314 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 19:26:09.317095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980377 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980382 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980386 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980390 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980393 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980396 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980398 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980401 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980404 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980407 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980409 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980412 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980414 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980417 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980420 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980424 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980428 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980431 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980434 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:26:09.317586 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980437 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980440 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980443 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980446 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980449 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980451 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980454 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980457 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980459 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980462 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980465 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980468 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980471 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980475 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980479 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980482 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980485 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980487 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980490 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:26:09.318153 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980492 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980495 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980497 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980500 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980503 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980505 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980508 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980513 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980516 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980518 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980521 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980524 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980527 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980529 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980532 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980536 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980539 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980541 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980544 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:26:09.318760 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980548 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980551 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980555 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980558 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980560 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980563 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980566 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980569 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980571 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980574 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980577 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980579 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980582 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980584 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980587 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980590 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980592 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980595 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980598 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980600 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980604 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:26:09.319698 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980607 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980610 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980612 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980615 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980617 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980620 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980622 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980626 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980629 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980631 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980634 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980636 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980639 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980642 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980644 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980647 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980649 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980653 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980655 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980658 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:26:09.320502 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980660 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.980665 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980780 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980785 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980788 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980791 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980793 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980796 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980799 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980801 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980804 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980806 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980811 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980816 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980818 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:26:09.321161 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980821 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980824 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980826 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980829 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980833 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980836 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980838 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980841 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980843 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980846 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980848 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980851 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980854 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980858 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980861 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980865 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980868 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980871 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980874 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:26:09.321823 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980876 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980880 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980883 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980886 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980889 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980892 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980895 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980897 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980900 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980902 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980907 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980909 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980912 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980914 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980917 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980920 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980922 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980926 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980929 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:26:09.322445 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980931 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980934 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980936 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980939 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980942 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980944 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980947 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980949 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980952 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980954 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980957 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980960 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980963 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980965 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980968 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980970 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980973 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980975 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980978 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980980 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:26:09.323074 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980983 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980985 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980988 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980992 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980995 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.980997 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981000 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981002 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981005 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981008 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981011 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981014 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981016 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981019 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981021 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981024 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981026 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981029 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981032 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981034 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:26:09.478737 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981037 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981039 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981042 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981045 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981048 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981050 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981053 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:08.981055 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.981061 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.981179 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.984466 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.985729 2565 server.go:1019] "Starting client certificate rotation" Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.985829 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:26:09.479541 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.985865 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:26:09.481024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.988178 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:26:09.481024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:08.989417 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:26:09.481024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.000364 2565 log.go:25] "Validated CRI v1 runtime API" Jul 06 19:26:09.481024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.002491 2565 log.go:25] "Validated CRI v1 image API" Jul 06 19:26:09.481024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.003305 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 19:26:09.481024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.009082 2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cf8fcb0d-133c-4bc5-948a-fdec44526f04:/dev/nvme0n1p4 ecb3cca8-862a-4637-b0ce-ab96608cf487:/dev/nvme0n1p3] Jul 06 19:26:09.481024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.009103 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.014760 2565 manager.go:216] Machine: {Timestamp:2026-07-06 19:26:09.013451081 +0000 UTC m=+0.387907013 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108657 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec267f5581b32c537f38a3ba970002df SystemUUID:ec267f55-81b3-2c53-7f38-a3ba970002df BootID:a576f037-1460-498c-8661-109d46d1dbae Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:33:d7:9f:fd:b9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:33:d7:9f:fd:b9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:52:e9:77:aa:e3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.014856 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.014936 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.018690 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.018730 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-100.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.018914 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.019619 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.019631 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.019639 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.019652 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.020993 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.023750 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.023855 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.023882 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.023890 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.023901 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 19:26:09.481351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.023910 2565 kubelet.go:398] "Adding apiserver pod source" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.023919 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.024364 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.024383 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.027309 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.032604 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.032636 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033844 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033860 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033866 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033872 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033881 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033886 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033894 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033899 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033906 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033911 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033920 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.033925 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.035661 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.035672 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.036679 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.036690 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-100.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.039282 2565 server.go:1268] "Started kubelet" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.039390 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.039438 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.039780 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.041051 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.042003 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.045107 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-100.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.045530 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.045691 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-djm7z" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.044985 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-100.ec2.internal.18bfc9e67117a5f7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-100.ec2.internal,UID:ip-10-0-143-100.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-100.ec2.internal,},FirstTimestamp:2026-07-06 19:26:09.039042039 +0000 UTC m=+0.413498018,LastTimestamp:2026-07-06 19:26:09.039042039 +0000 UTC m=+0.413498018,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-100.ec2.internal,}" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.048518 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.048626 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.048799 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049196 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049234 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049248 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049353 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 06 19:26:09.481901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049362 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049381 2565 factory.go:55] Registering systemd factory Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049393 2565 factory.go:223] Registration of the systemd container factory successfully Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049591 2565 factory.go:153] Registering CRI-O factory Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049605 2565 factory.go:223] Registration of the crio container factory successfully Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049646 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049669 2565 factory.go:103] Registering Raw factory Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.049681 2565 manager.go:1196] Started watching for new ooms in manager Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.050025 2565 manager.go:349] Starting recovery of all containers Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.054495 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.054637 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-100.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.057620 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-djm7z" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.058246 2565 manager.go:354] Recovery completed Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.065723 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.067520 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.067548 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.067558 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.068045 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.068053 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.068054 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.068074 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.068673 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.068689 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.068702 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.069309 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-100.ec2.internal.18bfc9e672ca6569 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-100.ec2.internal,UID:ip-10-0-143-100.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-100.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-100.ec2.internal,},FirstTimestamp:2026-07-06 19:26:09.067533673 +0000 UTC m=+0.441989603,LastTimestamp:2026-07-06 19:26:09.067533673 +0000 UTC m=+0.441989603,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-100.ec2.internal,}" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.073531 2565 policy_none.go:49] "None policy: Start" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.073545 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.073555 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.080396 2565 policy_none.go:47] "Start" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.149607 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.152848 2565 manager.go:337] "Starting Device Plugin manager" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.152885 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.152897 2565 server.go:87] "Starting device plugin registration server" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.153207 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.153235 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.153328 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.153422 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.153431 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.153864 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.153914 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.483018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.196494 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.197731 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.197767 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.197788 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.197820 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.200426 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.253839 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.255470 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.255503 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.255512 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.255537 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.264779 2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-100.ec2.internal\" not found" node="ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.266156 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.266173 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-100.ec2.internal\": node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.281724 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.310255 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal"] Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.310339 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.311353 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.311384 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.311393 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.312610 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.312725 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.312772 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.313334 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.313359 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.313364 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.313391 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.313405 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.313370 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.314653 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.314675 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.315373 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.315390 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.315399 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.353039 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-100.ec2.internal\" not found" node="ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.360065 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-100.ec2.internal\" not found" node="ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.382817 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.641265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.449915 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8131de443f47a89b4230bfcbf8311bf6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal\" (UID: \"8131de443f47a89b4230bfcbf8311bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.449954 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8131de443f47a89b4230bfcbf8311bf6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal\" (UID: \"8131de443f47a89b4230bfcbf8311bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.450006 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0903f6f852d654ad4e05c5ebda33a529-config\") pod \"kube-apiserver-proxy-ip-10-0-143-100.ec2.internal\" (UID: \"0903f6f852d654ad4e05c5ebda33a529\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.483057 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.550355 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0903f6f852d654ad4e05c5ebda33a529-config\") pod \"kube-apiserver-proxy-ip-10-0-143-100.ec2.internal\" (UID: \"0903f6f852d654ad4e05c5ebda33a529\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.550322 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0903f6f852d654ad4e05c5ebda33a529-config\") pod \"kube-apiserver-proxy-ip-10-0-143-100.ec2.internal\" (UID: \"0903f6f852d654ad4e05c5ebda33a529\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.550439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8131de443f47a89b4230bfcbf8311bf6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal\" (UID: \"8131de443f47a89b4230bfcbf8311bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.550462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8131de443f47a89b4230bfcbf8311bf6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal\" (UID: \"8131de443f47a89b4230bfcbf8311bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.550505 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8131de443f47a89b4230bfcbf8311bf6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal\" (UID: \"8131de443f47a89b4230bfcbf8311bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.550509 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8131de443f47a89b4230bfcbf8311bf6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal\" (UID: \"8131de443f47a89b4230bfcbf8311bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.642622 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.583449 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.797612 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.656864 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.797612 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.664040 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" Jul 06 19:26:09.797612 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.683687 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.797612 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.786234 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:09.969005 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.888278 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:10.107319 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.985544 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 19:26:10.107319 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:09.985721 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:26:10.107319 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:09.988956 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:10.107319 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.064717 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:21:09 +0000 UTC" deadline="2028-03-11 08:57:40.562791666 +0000 UTC" Jul 06 19:26:10.107319 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.064744 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14725h31m30.498050933s" Jul 06 19:26:10.107319 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:10.089613 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:10.202145 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:10.191048 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:10.324846 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:10.291986 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:10.324846 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:10.315898 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8131de443f47a89b4230bfcbf8311bf6.slice/crio-56a4cfd5db0f2279754e511758b40f0275df92b065795ace2d7eaf8480e1e018 WatchSource:0}: Error finding container 56a4cfd5db0f2279754e511758b40f0275df92b065795ace2d7eaf8480e1e018: Status 404 returned error can't find the container with id 56a4cfd5db0f2279754e511758b40f0275df92b065795ace2d7eaf8480e1e018 Jul 06 19:26:10.324846 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:10.316393 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0903f6f852d654ad4e05c5ebda33a529.slice/crio-f1cede1d620ec2c9271524251b2f4f0c44f075de13e6a964c264198bf5aae343 WatchSource:0}: Error finding container f1cede1d620ec2c9271524251b2f4f0c44f075de13e6a964c264198bf5aae343: Status 404 returned error can't find the container with id f1cede1d620ec2c9271524251b2f4f0c44f075de13e6a964c264198bf5aae343 Jul 06 19:26:10.324846 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.320720 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:26:10.491106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.371416 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:26:10.491106 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:10.392960 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-100.ec2.internal\" not found" Jul 06 19:26:10.491106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.407815 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:26:10.491106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.450715 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" Jul 06 19:26:10.491106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.455028 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:26:10.491106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.469656 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:26:10.491106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.470788 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" Jul 06 19:26:10.491106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.487234 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:26:10.917574 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:10.909614 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.027504 2565 apiserver.go:52] "Watching apiserver" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.040478 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.042152 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-5lcpf","kube-system/konnectivity-agent-wk8mp","kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal","openshift-image-registry/node-ca-2pdr9","openshift-network-diagnostics/network-check-target-gndv4","openshift-network-operator/iptables-alerter-9lclq","openshift-ovn-kubernetes/ovnkube-node-9ls98","kube-system/global-pull-secret-syncer-b6ks2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw","openshift-cluster-node-tuning-operator/tuned-8kq4x","openshift-dns/node-resolver-blvmv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal","openshift-multus/multus-additional-cni-plugins-wf722","openshift-multus/multus-cksts"] Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.045019 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.047234 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.049020 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cksts" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.049304 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.050698 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.052789 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.052845 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.056020 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.057110 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.058438 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.058526 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.059750 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060400 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-cnibin\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060435 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-socket-dir-parent\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060466 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-netns\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060510 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-daemon-config\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060539 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-node-log\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060564 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-slash\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.084690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060586 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-run-netns\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060625 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-cni-multus\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060649 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-var-lib-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060666 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-env-overrides\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060689 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-ovnkube-script-lib\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060719 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060759 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060785 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-multus-certs\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060820 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77d302df-429b-47c4-a97c-c8097c1db9fd-host\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060852 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77d302df-429b-47c4-a97c-c8097c1db9fd-serviceca\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-log-socket\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060913 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c96v5\" (UniqueName: \"kubernetes.io/projected/805cce93-10de-4cb9-a20e-29561db3c966-kube-api-access-c96v5\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060975 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-conf-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.060990 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74dvj\" (UniqueName: \"kubernetes.io/projected/77d302df-429b-47c4-a97c-c8097c1db9fd-kube-api-access-74dvj\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061022 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-systemd-units\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.086133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061040 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-cni-bin\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061060 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-cni-netd\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061087 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/805cce93-10de-4cb9-a20e-29561db3c966-ovn-node-metrics-cert\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061138 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061160 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fa741bb-09fc-450d-950e-f4099a3b8278-cni-binary-copy\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-k8s-cni-cncf-io\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061334 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-cni-bin\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061371 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-os-release\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061399 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-kubelet\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061375 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061463 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-ovnkube-config\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061488 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-system-cni-dir\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061515 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phtlv\" (UniqueName: \"kubernetes.io/projected/51d20679-22ec-4bdf-802a-2b15999befec-kube-api-access-phtlv\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061545 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-os-release\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061592 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-cni-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.088629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061625 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-etc-kubernetes\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061665 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-etc-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-ovn\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061717 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-kubelet\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061741 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-hostroot\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061765 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfwll\" (UniqueName: \"kubernetes.io/projected/2fa741bb-09fc-450d-950e-f4099a3b8278-kube-api-access-jfwll\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-systemd\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061821 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-cnibin\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.061881 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-system-cni-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.062820 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.064899 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.090036 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.064960 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:11.094620 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.092676 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:26:11.094620 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.094274 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:26:11.094620 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.094364 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:26:11.094620 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.094545 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qtdsm\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.095290 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.095488 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.095737 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.095889 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jt8s4\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.096024 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.096127 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.096240 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.096408 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.096584 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jwdjk\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.096742 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:26:11.098399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.096863 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:26:11.101495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.100524 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:26:11.101495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.100712 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h2sqw\"" Jul 06 19:26:11.101495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.100872 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:26:11.101495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.101013 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:26:11.101495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.101146 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:26:11.101495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.101299 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bjtmx\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.101597 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fshrh\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.102012 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.102241 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gb97f\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.102401 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.102576 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.102829 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.102986 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k5wcw\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.103138 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.103346 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:26:11.103628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.103475 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:26:11.104570 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.103705 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:26:11.104570 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.103828 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:26:11.104570 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.103966 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:26:11.104570 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.104067 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j6q6c\"" Jul 06 19:26:11.104570 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.104223 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.150949 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.162826 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-run-netns\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.162862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-cni-multus\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.162886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wngmh\" (UniqueName: \"kubernetes.io/projected/048d362e-572e-45c4-8d5a-ee18eb72e278-kube-api-access-wngmh\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.162907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-env-overrides\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.162934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-multus-certs\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.162960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77d302df-429b-47c4-a97c-c8097c1db9fd-host\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.162986 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7a726848-72dc-4069-965a-d61c7f72a321-konnectivity-ca\") pod \"konnectivity-agent-wk8mp\" (UID: \"7a726848-72dc-4069-965a-d61c7f72a321\") " pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163013 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-tuned\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163039 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-log-socket\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163092 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-conf-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163117 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-sys-fs\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163134 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7a726848-72dc-4069-965a-d61c7f72a321-agent-certs\") pod \"konnectivity-agent-wk8mp\" (UID: \"7a726848-72dc-4069-965a-d61c7f72a321\") " pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163151 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-modprobe-d\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-host\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163195 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/327e6a13-74c5-4ce6-8d39-79b9143bbc44-tmp\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.180123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-systemd-units\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163233 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-cni-netd\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163249 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/805cce93-10de-4cb9-a20e-29561db3c966-ovn-node-metrics-cert\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163285 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163311 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fa741bb-09fc-450d-950e-f4099a3b8278-cni-binary-copy\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163364 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163390 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-etc-kubernetes\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163432 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxpkq\" (UniqueName: \"kubernetes.io/projected/d1df10a1-d344-41da-a87f-1df534ffc631-kube-api-access-gxpkq\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163448 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/17bca18e-95ba-4d4b-b844-7f7b52c769f2-dbus\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163470 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163525 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-cni-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163549 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-systemd\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163578 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-ovn\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163603 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/048d362e-572e-45c4-8d5a-ee18eb72e278-host-slash\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.181257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163628 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-etc-selinux\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-systemd\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-cnibin\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163710 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-cnibin\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-netns\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163765 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/048d362e-572e-45c4-8d5a-ee18eb72e278-iptables-alerter-script\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysctl-d\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163816 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysctl-conf\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163842 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-node-log\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163866 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-device-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163896 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysconfig\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163934 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-lib-modules\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163966 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-var-lib-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.163993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-ovnkube-script-lib\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164024 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164077 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77d302df-429b-47c4-a97c-c8097c1db9fd-serviceca\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.183547 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164103 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-sys\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164129 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-var-lib-kubelet\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164155 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c96v5\" (UniqueName: \"kubernetes.io/projected/805cce93-10de-4cb9-a20e-29561db3c966-kube-api-access-c96v5\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74dvj\" (UniqueName: \"kubernetes.io/projected/77d302df-429b-47c4-a97c-c8097c1db9fd-kube-api-access-74dvj\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164228 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-run\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9rj5\" (UniqueName: \"kubernetes.io/projected/327e6a13-74c5-4ce6-8d39-79b9143bbc44-kube-api-access-q9rj5\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxhqt\" (UniqueName: \"kubernetes.io/projected/14de08b3-3e7b-4641-97c8-7140cb4c24ec-kube-api-access-nxhqt\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164342 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-cni-bin\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164373 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-k8s-cni-cncf-io\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164399 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-cni-bin\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-kubelet\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwll\" (UniqueName: \"kubernetes.io/projected/2fa741bb-09fc-450d-950e-f4099a3b8278-kube-api-access-jfwll\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-os-release\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-kubelet\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164536 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-ovnkube-config\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164562 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-system-cni-dir\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phtlv\" (UniqueName: \"kubernetes.io/projected/51d20679-22ec-4bdf-802a-2b15999befec-kube-api-access-phtlv\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.184693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164616 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-os-release\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164658 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-registration-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164717 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-socket-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw4h8\" (UniqueName: \"kubernetes.io/projected/84c5d61e-9eee-410f-9591-0e6932cd4a34-kube-api-access-fw4h8\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164781 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1df10a1-d344-41da-a87f-1df534ffc631-hosts-file\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164807 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1df10a1-d344-41da-a87f-1df534ffc631-tmp-dir\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/17bca18e-95ba-4d4b-b844-7f7b52c769f2-kubelet-config\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164878 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-etc-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-hostroot\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.164962 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-system-cni-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.165196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-socket-dir-parent\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.165711 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-daemon-config\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.166255 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.167113 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-kubernetes\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.167375 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-slash\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.167497 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-slash\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.187262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.167553 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-run-netns\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.167603 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-cni-multus\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.168182 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-env-overrides\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.168264 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-multus-certs\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.168342 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77d302df-429b-47c4-a97c-c8097c1db9fd-host\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.168412 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-log-socket\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.168710 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-var-lib-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.170871 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.171956 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.172651 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.172843 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-cni-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.172882 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-conf-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.172930 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-ovn\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.173427 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-run-systemd\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.173724 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-systemd-units\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.173912 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-cnibin\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.174174 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.174215 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-cni-netd\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.188433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.174300 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-cnibin\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.174684 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-node-log\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.174733 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-netns\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.174948 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-kubelet\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.175334 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-system-cni-dir\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.175522 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.175869 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2fa741bb-09fc-450d-950e-f4099a3b8278-cni-binary-copy\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176212 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-ovnkube-script-lib\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176280 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-etc-kubernetes\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176350 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-etc-openvswitch\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176424 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51d20679-22ec-4bdf-802a-2b15999befec-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176430 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-cni-bin\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176460 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176518 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-socket-dir-parent\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176611 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-kubelet\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176693 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-var-lib-cni-bin\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176752 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-hostroot\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176821 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-system-cni-dir\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.193202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.176859 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77d302df-429b-47c4-a97c-c8097c1db9fd-serviceca\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.194640 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.177336 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-host-run-k8s-cni-cncf-io\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.194640 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.177397 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805cce93-10de-4cb9-a20e-29561db3c966-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.194640 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.177538 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/805cce93-10de-4cb9-a20e-29561db3c966-ovnkube-config\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.194640 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.177563 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51d20679-22ec-4bdf-802a-2b15999befec-os-release\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.194640 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.177667 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2fa741bb-09fc-450d-950e-f4099a3b8278-os-release\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.194640 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.178577 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2fa741bb-09fc-450d-950e-f4099a3b8278-multus-daemon-config\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.194640 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.182004 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/805cce93-10de-4cb9-a20e-29561db3c966-ovn-node-metrics-cert\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.257174 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.209979 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" event={"ID":"8131de443f47a89b4230bfcbf8311bf6","Type":"ContainerStarted","Data":"56a4cfd5db0f2279754e511758b40f0275df92b065795ace2d7eaf8480e1e018"} Jul 06 19:26:11.257174 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.211760 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" event={"ID":"0903f6f852d654ad4e05c5ebda33a529","Type":"ContainerStarted","Data":"f1cede1d620ec2c9271524251b2f4f0c44f075de13e6a964c264198bf5aae343"} Jul 06 19:26:11.266472 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.266397 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:26:11.266472 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.266442 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gmp4n" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-device-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267735 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysconfig\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267762 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-lib-modules\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-sys\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267815 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-var-lib-kubelet\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267832 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-device-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267838 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-run\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267891 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-run\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267895 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9rj5\" (UniqueName: \"kubernetes.io/projected/327e6a13-74c5-4ce6-8d39-79b9143bbc44-kube-api-access-q9rj5\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxhqt\" (UniqueName: \"kubernetes.io/projected/14de08b3-3e7b-4641-97c8-7140cb4c24ec-kube-api-access-nxhqt\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:11.267979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267948 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysconfig\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.267987 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-registration-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-socket-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw4h8\" (UniqueName: \"kubernetes.io/projected/84c5d61e-9eee-410f-9591-0e6932cd4a34-kube-api-access-fw4h8\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268086 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1df10a1-d344-41da-a87f-1df534ffc631-hosts-file\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1df10a1-d344-41da-a87f-1df534ffc631-tmp-dir\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268133 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/17bca18e-95ba-4d4b-b844-7f7b52c769f2-kubelet-config\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268164 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268189 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-kubernetes\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268219 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngmh\" (UniqueName: \"kubernetes.io/projected/048d362e-572e-45c4-8d5a-ee18eb72e278-kube-api-access-wngmh\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7a726848-72dc-4069-965a-d61c7f72a321-konnectivity-ca\") pod \"konnectivity-agent-wk8mp\" (UID: \"7a726848-72dc-4069-965a-d61c7f72a321\") " pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268320 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-tuned\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268348 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-sys-fs\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268377 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7a726848-72dc-4069-965a-d61c7f72a321-agent-certs\") pod \"konnectivity-agent-wk8mp\" (UID: \"7a726848-72dc-4069-965a-d61c7f72a321\") " pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268401 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-modprobe-d\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268426 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-host\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268450 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/327e6a13-74c5-4ce6-8d39-79b9143bbc44-tmp\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.268780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268483 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268519 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268555 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxpkq\" (UniqueName: \"kubernetes.io/projected/d1df10a1-d344-41da-a87f-1df534ffc631-kube-api-access-gxpkq\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/17bca18e-95ba-4d4b-b844-7f7b52c769f2-dbus\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268609 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268639 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-systemd\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/048d362e-572e-45c4-8d5a-ee18eb72e278-host-slash\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268705 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-etc-selinux\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268740 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/048d362e-572e-45c4-8d5a-ee18eb72e278-iptables-alerter-script\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268767 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysctl-d\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysctl-conf\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268849 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-lib-modules\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268909 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-sys\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268954 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-var-lib-kubelet\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.268971 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysctl-conf\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.269008 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-registration-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.269071 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-sys-fs\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.269872 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.269108 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-socket-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.271438 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.270425 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/17bca18e-95ba-4d4b-b844-7f7b52c769f2-dbus\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.271438 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.270532 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-modprobe-d\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.271438 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.270581 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-host\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.271438 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.270668 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfwll\" (UniqueName: \"kubernetes.io/projected/2fa741bb-09fc-450d-950e-f4099a3b8278-kube-api-access-jfwll\") pod \"multus-cksts\" (UID: \"2fa741bb-09fc-450d-950e-f4099a3b8278\") " pod="openshift-multus/multus-cksts" Jul 06 19:26:11.271438 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.270818 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:11.271438 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.270893 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:26:11.770871421 +0000 UTC m=+3.145327346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:11.271438 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.271381 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/17bca18e-95ba-4d4b-b844-7f7b52c769f2-kubelet-config\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.271805 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.271703 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1df10a1-d344-41da-a87f-1df534ffc631-hosts-file\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.272204 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.272074 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1df10a1-d344-41da-a87f-1df534ffc631-tmp-dir\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.272204 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.272115 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-etc-selinux\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.272204 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.272157 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-systemd\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.272204 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.272170 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/048d362e-572e-45c4-8d5a-ee18eb72e278-host-slash\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.272473 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.272229 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:11.272473 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.272275 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret podName:17bca18e-95ba-4d4b-b844-7f7b52c769f2 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:11.772259843 +0000 UTC m=+3.146715777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret") pod "global-pull-secret-syncer-b6ks2" (UID: "17bca18e-95ba-4d4b-b844-7f7b52c769f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.272787 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/048d362e-572e-45c4-8d5a-ee18eb72e278-iptables-alerter-script\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.272872 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7a726848-72dc-4069-965a-d61c7f72a321-konnectivity-ca\") pod \"konnectivity-agent-wk8mp\" (UID: \"7a726848-72dc-4069-965a-d61c7f72a321\") " pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.272929 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-sysctl-d\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.272980 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-kubernetes\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.273023 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84c5d61e-9eee-410f-9591-0e6932cd4a34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.274328 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c96v5\" (UniqueName: \"kubernetes.io/projected/805cce93-10de-4cb9-a20e-29561db3c966-kube-api-access-c96v5\") pod \"ovnkube-node-9ls98\" (UID: \"805cce93-10de-4cb9-a20e-29561db3c966\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.274967 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/327e6a13-74c5-4ce6-8d39-79b9143bbc44-etc-tuned\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.275615 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/327e6a13-74c5-4ce6-8d39-79b9143bbc44-tmp\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.275925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.275865 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phtlv\" (UniqueName: \"kubernetes.io/projected/51d20679-22ec-4bdf-802a-2b15999befec-kube-api-access-phtlv\") pod \"multus-additional-cni-plugins-wf722\" (UID: \"51d20679-22ec-4bdf-802a-2b15999befec\") " pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.276381 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.276351 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74dvj\" (UniqueName: \"kubernetes.io/projected/77d302df-429b-47c4-a97c-c8097c1db9fd-kube-api-access-74dvj\") pod \"node-ca-2pdr9\" (UID: \"77d302df-429b-47c4-a97c-c8097c1db9fd\") " pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.277743 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.277724 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7a726848-72dc-4069-965a-d61c7f72a321-agent-certs\") pod \"konnectivity-agent-wk8mp\" (UID: \"7a726848-72dc-4069-965a-d61c7f72a321\") " pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:11.280190 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.280162 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9rj5\" (UniqueName: \"kubernetes.io/projected/327e6a13-74c5-4ce6-8d39-79b9143bbc44-kube-api-access-q9rj5\") pod \"tuned-8kq4x\" (UID: \"327e6a13-74c5-4ce6-8d39-79b9143bbc44\") " pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.280388 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.280369 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxhqt\" (UniqueName: \"kubernetes.io/projected/14de08b3-3e7b-4641-97c8-7140cb4c24ec-kube-api-access-nxhqt\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:11.281335 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.281314 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:26:11.281335 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.281337 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:26:11.281499 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.281347 2565 projected.go:196] Error preparing data for projected volume kube-api-access-dzq8t for pod openshift-network-diagnostics/network-check-target-gndv4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:11.281499 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.281400 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t podName:e7cbcf2f-c466-4465-ad91-5094d5bb4cb9 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:11.781386689 +0000 UTC m=+3.155842626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dzq8t" (UniqueName: "kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t") pod "network-check-target-gndv4" (UID: "e7cbcf2f-c466-4465-ad91-5094d5bb4cb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:11.282390 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.282355 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw4h8\" (UniqueName: \"kubernetes.io/projected/84c5d61e-9eee-410f-9591-0e6932cd4a34-kube-api-access-fw4h8\") pod \"aws-ebs-csi-driver-node-plmgw\" (UID: \"84c5d61e-9eee-410f-9591-0e6932cd4a34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.285300 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.285244 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wngmh\" (UniqueName: \"kubernetes.io/projected/048d362e-572e-45c4-8d5a-ee18eb72e278-kube-api-access-wngmh\") pod \"iptables-alerter-9lclq\" (UID: \"048d362e-572e-45c4-8d5a-ee18eb72e278\") " pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.289034 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.289010 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxpkq\" (UniqueName: \"kubernetes.io/projected/d1df10a1-d344-41da-a87f-1df534ffc631-kube-api-access-gxpkq\") pod \"node-resolver-blvmv\" (UID: \"d1df10a1-d344-41da-a87f-1df534ffc631\") " pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.308000 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.307835 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9lclq" Jul 06 19:26:11.319018 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.318998 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" Jul 06 19:26:11.327278 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.326993 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:11.329906 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:11.329871 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048d362e_572e_45c4_8d5a_ee18eb72e278.slice/crio-731d5b51ce09be28e099c54b7fd6e01b6dc7b1b0b22b83e5c82480dd36cf6425 WatchSource:0}: Error finding container 731d5b51ce09be28e099c54b7fd6e01b6dc7b1b0b22b83e5c82480dd36cf6425: Status 404 returned error can't find the container with id 731d5b51ce09be28e099c54b7fd6e01b6dc7b1b0b22b83e5c82480dd36cf6425 Jul 06 19:26:11.336601 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.336579 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" Jul 06 19:26:11.345116 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.345096 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-blvmv" Jul 06 19:26:11.353352 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:11.353263 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327e6a13_74c5_4ce6_8d39_79b9143bbc44.slice/crio-902e005046183fd7a8ebfb3a614065c7d4ce185cc390264b87edda5f57fc1c25 WatchSource:0}: Error finding container 902e005046183fd7a8ebfb3a614065c7d4ce185cc390264b87edda5f57fc1c25: Status 404 returned error can't find the container with id 902e005046183fd7a8ebfb3a614065c7d4ce185cc390264b87edda5f57fc1c25 Jul 06 19:26:11.359869 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.359769 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:11.368456 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.368428 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wf722" Jul 06 19:26:11.379688 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:11.379659 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d20679_22ec_4bdf_802a_2b15999befec.slice/crio-9ad58632f90e4102844b00a5d38c94824264812634f84cb0a4dea1995be77aeb WatchSource:0}: Error finding container 9ad58632f90e4102844b00a5d38c94824264812634f84cb0a4dea1995be77aeb: Status 404 returned error can't find the container with id 9ad58632f90e4102844b00a5d38c94824264812634f84cb0a4dea1995be77aeb Jul 06 19:26:11.381255 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.381219 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cksts" Jul 06 19:26:11.384529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.384359 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pdr9" Jul 06 19:26:11.396035 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:11.395993 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa741bb_09fc_450d_950e_f4099a3b8278.slice/crio-7fd0398910909d29e4f846c2fc1015da4692d4a64f58507a1571df1861e68cf4 WatchSource:0}: Error finding container 7fd0398910909d29e4f846c2fc1015da4692d4a64f58507a1571df1861e68cf4: Status 404 returned error can't find the container with id 7fd0398910909d29e4f846c2fc1015da4692d4a64f58507a1571df1861e68cf4 Jul 06 19:26:11.400075 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:11.399889 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d302df_429b_47c4_a97c_c8097c1db9fd.slice/crio-26b8a7fa37bafaf3eb335ed112d9945ebac4317549029d07cb245bb1ce1b66ef WatchSource:0}: Error finding container 26b8a7fa37bafaf3eb335ed112d9945ebac4317549029d07cb245bb1ce1b66ef: Status 404 returned error can't find the container with id 26b8a7fa37bafaf3eb335ed112d9945ebac4317549029d07cb245bb1ce1b66ef Jul 06 19:26:11.774637 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.774601 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:11.774637 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.774643 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:11.774889 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.774805 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:11.774889 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.774881 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:26:12.774860444 +0000 UTC m=+4.149316385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:11.774996 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.774808 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:11.774996 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.774958 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret podName:17bca18e-95ba-4d4b-b844-7f7b52c769f2 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:12.774938876 +0000 UTC m=+4.149394821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret") pod "global-pull-secret-syncer-b6ks2" (UID: "17bca18e-95ba-4d4b-b844-7f7b52c769f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:11.875619 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:11.875575 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:11.875791 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.875777 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:26:11.875837 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.875803 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:26:11.875837 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.875814 2565 projected.go:196] Error preparing data for projected volume kube-api-access-dzq8t for pod openshift-network-diagnostics/network-check-target-gndv4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:11.875904 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:11.875869 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t podName:e7cbcf2f-c466-4465-ad91-5094d5bb4cb9 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:12.875854664 +0000 UTC m=+4.250310601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dzq8t" (UniqueName: "kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t") pod "network-check-target-gndv4" (UID: "e7cbcf2f-c466-4465-ad91-5094d5bb4cb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:12.198338 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.198236 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:12.199071 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.198398 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:12.216068 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.216030 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cksts" event={"ID":"2fa741bb-09fc-450d-950e-f4099a3b8278","Type":"ContainerStarted","Data":"7fd0398910909d29e4f846c2fc1015da4692d4a64f58507a1571df1861e68cf4"} Jul 06 19:26:12.217877 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.217845 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf722" event={"ID":"51d20679-22ec-4bdf-802a-2b15999befec","Type":"ContainerStarted","Data":"9ad58632f90e4102844b00a5d38c94824264812634f84cb0a4dea1995be77aeb"} Jul 06 19:26:12.219504 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.219397 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"8d2f5c2d3e60f88f6c95c019f36e3038d7f49e41537b6762ea2c15f43aa25845"} Jul 06 19:26:12.220737 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.220658 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-blvmv" event={"ID":"d1df10a1-d344-41da-a87f-1df534ffc631","Type":"ContainerStarted","Data":"932b60317bde5966f565f6c9ea410959e1d2396aab0772191510355117493317"} Jul 06 19:26:12.222616 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.222531 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wk8mp" event={"ID":"7a726848-72dc-4069-965a-d61c7f72a321","Type":"ContainerStarted","Data":"3dfa06455b0bb1cc2b82e8fdd5e50a58ba808f3bfe218663a4d07c4b11c9bb01"} Jul 06 19:26:12.223786 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.223730 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" event={"ID":"327e6a13-74c5-4ce6-8d39-79b9143bbc44","Type":"ContainerStarted","Data":"902e005046183fd7a8ebfb3a614065c7d4ce185cc390264b87edda5f57fc1c25"} Jul 06 19:26:12.225693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.225661 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" event={"ID":"84c5d61e-9eee-410f-9591-0e6932cd4a34","Type":"ContainerStarted","Data":"5e66195b3321e73235dc63a1b92f54a5870d131d73c87108d7a78c65d77bc6b6"} Jul 06 19:26:12.227501 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.227476 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9lclq" event={"ID":"048d362e-572e-45c4-8d5a-ee18eb72e278","Type":"ContainerStarted","Data":"731d5b51ce09be28e099c54b7fd6e01b6dc7b1b0b22b83e5c82480dd36cf6425"} Jul 06 19:26:12.230821 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.230756 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pdr9" event={"ID":"77d302df-429b-47c4-a97c-c8097c1db9fd","Type":"ContainerStarted","Data":"26b8a7fa37bafaf3eb335ed112d9945ebac4317549029d07cb245bb1ce1b66ef"} Jul 06 19:26:12.272875 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.272821 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:21:11 +0000 UTC" deadline="2028-03-18 12:55:08.602939184 +0000 UTC" Jul 06 19:26:12.272875 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.272867 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14897h28m56.330078455s" Jul 06 19:26:12.785418 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.785382 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:12.785609 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.785432 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:12.785609 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.785574 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:12.785751 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.785640 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret podName:17bca18e-95ba-4d4b-b844-7f7b52c769f2 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:14.785620378 +0000 UTC m=+6.160076297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret") pod "global-pull-secret-syncer-b6ks2" (UID: "17bca18e-95ba-4d4b-b844-7f7b52c769f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:12.785751 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.785729 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:12.785864 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.785772 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:26:14.785758224 +0000 UTC m=+6.160214150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:12.886674 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:12.886629 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:12.887016 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.886992 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:26:12.887111 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.887027 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:26:12.887111 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.887041 2565 projected.go:196] Error preparing data for projected volume kube-api-access-dzq8t for pod openshift-network-diagnostics/network-check-target-gndv4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:12.887111 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:12.887109 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t podName:e7cbcf2f-c466-4465-ad91-5094d5bb4cb9 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:14.887088002 +0000 UTC m=+6.261543946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dzq8t" (UniqueName: "kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t") pod "network-check-target-gndv4" (UID: "e7cbcf2f-c466-4465-ad91-5094d5bb4cb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:13.199050 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:13.198966 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:13.199552 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:13.199103 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:13.199552 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:13.199547 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:13.199676 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:13.199654 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:13.273479 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:13.273437 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:21:11 +0000 UTC" deadline="2028-01-07 08:58:26.206527641 +0000 UTC" Jul 06 19:26:13.273479 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:13.273478 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13189h32m12.933054817s" Jul 06 19:26:14.199927 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:14.199379 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:14.199927 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.199540 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:14.805314 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:14.805276 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:14.805495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:14.805331 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:14.805495 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.805481 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:14.805609 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.805557 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret podName:17bca18e-95ba-4d4b-b844-7f7b52c769f2 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:18.805536706 +0000 UTC m=+10.179992631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret") pod "global-pull-secret-syncer-b6ks2" (UID: "17bca18e-95ba-4d4b-b844-7f7b52c769f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:14.805670 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.805626 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:14.805670 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.805662 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:26:18.805649748 +0000 UTC m=+10.180105673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:14.906714 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:14.906678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:14.906891 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.906860 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:26:14.906891 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.906880 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:26:14.906891 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.906892 2565 projected.go:196] Error preparing data for projected volume kube-api-access-dzq8t for pod openshift-network-diagnostics/network-check-target-gndv4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:14.907024 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:14.906952 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t podName:e7cbcf2f-c466-4465-ad91-5094d5bb4cb9 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:18.9069327 +0000 UTC m=+10.281388638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dzq8t" (UniqueName: "kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t") pod "network-check-target-gndv4" (UID: "e7cbcf2f-c466-4465-ad91-5094d5bb4cb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:15.200315 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:15.200238 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:15.200709 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:15.200374 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:15.200761 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:15.200239 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:15.200836 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:15.200814 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:16.199176 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:16.199068 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:16.199376 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:16.199237 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:17.198736 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:17.198251 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:17.198736 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:17.198275 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:17.198736 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:17.198389 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:17.198736 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:17.198525 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:18.198186 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:18.198144 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:18.198395 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.198363 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:18.841076 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:18.841038 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:18.841076 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:18.841087 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:18.841863 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.841270 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:18.841863 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.841327 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:18.841863 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.841341 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret podName:17bca18e-95ba-4d4b-b844-7f7b52c769f2 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:26.841320494 +0000 UTC m=+18.215776429 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret") pod "global-pull-secret-syncer-b6ks2" (UID: "17bca18e-95ba-4d4b-b844-7f7b52c769f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:18.841863 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.841367 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:26:26.841356096 +0000 UTC m=+18.215812028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:18.942338 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:18.941645 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:18.942338 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.941860 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:26:18.942338 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.941883 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:26:18.942338 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.941898 2565 projected.go:196] Error preparing data for projected volume kube-api-access-dzq8t for pod openshift-network-diagnostics/network-check-target-gndv4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:18.942338 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:18.941958 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t podName:e7cbcf2f-c466-4465-ad91-5094d5bb4cb9 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:26.941941327 +0000 UTC m=+18.316397255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dzq8t" (UniqueName: "kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t") pod "network-check-target-gndv4" (UID: "e7cbcf2f-c466-4465-ad91-5094d5bb4cb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:19.212715 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:19.210478 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:19.212715 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:19.210604 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:19.212715 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:19.210199 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:19.212715 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:19.211293 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:20.198304 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:20.198257 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:20.198709 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:20.198427 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:21.198976 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:21.198942 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:21.199383 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:21.199076 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:21.199763 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:21.199597 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:21.199763 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:21.199719 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:22.198787 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:22.198749 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:22.198938 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:22.198909 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:23.198329 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:23.198293 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:23.198329 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:23.198293 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:23.198787 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:23.198466 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:23.198787 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:23.198477 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:24.198033 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:24.197999 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:24.198247 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:24.198129 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:25.198697 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:25.198661 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:25.199153 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:25.198782 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:25.199153 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:25.198849 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:25.199153 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:25.198966 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:26.198944 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:26.198901 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:26.199363 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:26.199075 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:26.908128 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:26.908083 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:26.908301 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:26.908140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:26.908301 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:26.908243 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:26.908384 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:26.908312 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:26:42.908295377 +0000 UTC m=+34.282751297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:26.908384 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:26.908243 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:26.908490 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:26.908435 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret podName:17bca18e-95ba-4d4b-b844-7f7b52c769f2 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:42.908411866 +0000 UTC m=+34.282867787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret") pod "global-pull-secret-syncer-b6ks2" (UID: "17bca18e-95ba-4d4b-b844-7f7b52c769f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:27.008813 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:27.008780 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:27.008985 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:27.008953 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:26:27.008985 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:27.008981 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:26:27.009092 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:27.008992 2565 projected.go:196] Error preparing data for projected volume kube-api-access-dzq8t for pod openshift-network-diagnostics/network-check-target-gndv4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:27.009092 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:27.009056 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t podName:e7cbcf2f-c466-4465-ad91-5094d5bb4cb9 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:43.009036649 +0000 UTC m=+34.383492590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dzq8t" (UniqueName: "kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t") pod "network-check-target-gndv4" (UID: "e7cbcf2f-c466-4465-ad91-5094d5bb4cb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:27.198915 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:27.198829 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:27.199076 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:27.198967 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:27.199076 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:27.199026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:27.199505 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:27.199142 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:28.198421 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:28.198386 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:28.198594 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:28.198515 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:29.199807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:29.199565 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:29.200273 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:29.199640 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:29.200273 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:29.199908 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:29.200273 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:29.199953 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:30.198932 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:30.198900 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:30.199121 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:30.199021 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:31.198838 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.198759 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:31.198838 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.198791 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:31.199349 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:31.198900 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:31.199349 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:31.199027 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:31.274028 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.273344 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" event={"ID":"0903f6f852d654ad4e05c5ebda33a529","Type":"ContainerStarted","Data":"9b214d702dd913abb015495b83fbc436a603bd1ef3291aee078c8c11b7eef092"} Jul 06 19:26:31.274787 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.274765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cksts" event={"ID":"2fa741bb-09fc-450d-950e-f4099a3b8278","Type":"ContainerStarted","Data":"776fd1ba54453f8a6fb6032b35bbab2f2231847cb007d2c1fa9c4f9e54ddf30f"} Jul 06 19:26:31.276355 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.276331 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"51b3cfebe9775581edd8987c0b9a6ec99028e282dcdad5d91a9734012826423c"} Jul 06 19:26:31.276443 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.276394 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"89c42723c2b99007cad16a24a6910047f218c1948d390cbcc0cd6ff5c8f23659"} Jul 06 19:26:31.280698 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.280676 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" event={"ID":"327e6a13-74c5-4ce6-8d39-79b9143bbc44","Type":"ContainerStarted","Data":"25a3d06b39f30e28422a2480ab80c7d3f49fdc98da9eabf03db65ec2557ed168"} Jul 06 19:26:31.302554 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.302508 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-100.ec2.internal" podStartSLOduration=21.302493606 podStartE2EDuration="21.302493606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:31.301818722 +0000 UTC m=+22.676274662" watchObservedRunningTime="2026-07-06 19:26:31.302493606 +0000 UTC m=+22.676949547" Jul 06 19:26:31.321544 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:31.321500 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8kq4x" podStartSLOduration=3.274683323 podStartE2EDuration="22.321483319s" totalImagesPullingTime="19.046799996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:31.321328227 +0000 UTC m=+22.695784179" watchObservedRunningTime="2026-07-06 19:26:31.321483319 +0000 UTC m=+22.695939259" Jul 06 19:26:32.198411 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.198207 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:32.198641 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:32.198522 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:32.284480 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.284444 2565 generic.go:356] "Generic (PLEG): container finished" podID="8131de443f47a89b4230bfcbf8311bf6" containerID="5a7fabcb21e1b62e34ba6873d98d51e93adb1c621ac969464a982e3d71095c17" exitCode=0 Jul 06 19:26:32.285378 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.284527 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" event={"ID":"8131de443f47a89b4230bfcbf8311bf6","Type":"ContainerDied","Data":"5a7fabcb21e1b62e34ba6873d98d51e93adb1c621ac969464a982e3d71095c17"} Jul 06 19:26:32.286044 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.286013 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pdr9" event={"ID":"77d302df-429b-47c4-a97c-c8097c1db9fd","Type":"ContainerStarted","Data":"32d39cdada13a4e28efb44d979312e5a2dbc7f4a5d02bb240c8b1a654b73d74c"} Jul 06 19:26:32.287513 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.287491 2565 generic.go:356] "Generic (PLEG): container finished" podID="51d20679-22ec-4bdf-802a-2b15999befec" containerID="cb39be64daca3523a566fae0989cd3d14ab75acfc92035317ce9fb4d8b5267a8" exitCode=0 Jul 06 19:26:32.287613 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.287574 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf722" event={"ID":"51d20679-22ec-4bdf-802a-2b15999befec","Type":"ContainerDied","Data":"cb39be64daca3523a566fae0989cd3d14ab75acfc92035317ce9fb4d8b5267a8"} Jul 06 19:26:32.290404 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.290328 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:26:32.290715 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.290696 2565 generic.go:356] "Generic (PLEG): container finished" podID="805cce93-10de-4cb9-a20e-29561db3c966" containerID="51b3cfebe9775581edd8987c0b9a6ec99028e282dcdad5d91a9734012826423c" exitCode=1 Jul 06 19:26:32.290780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.290763 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerDied","Data":"51b3cfebe9775581edd8987c0b9a6ec99028e282dcdad5d91a9734012826423c"} Jul 06 19:26:32.290854 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.290827 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"c46176ccc81509e906d59351a9e6ec07eb39e10b4392ae412d19e53cf3b27c68"} Jul 06 19:26:32.290892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.290874 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"47fe49249ab18905ed07b169e48ee6d648acdccba46b0f4836224a794f08c6fd"} Jul 06 19:26:32.290934 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.290903 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"46de8b1e83ec06f1b5fdecd7ec20dc85cced414b1a3e84adf2e945620251ac96"} Jul 06 19:26:32.290977 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.290947 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"fc25c630ea39a873aaf8876f4d46511ef5f47e5a78927f0410b5498808495384"} Jul 06 19:26:32.292062 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.292040 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-blvmv" event={"ID":"d1df10a1-d344-41da-a87f-1df534ffc631","Type":"ContainerStarted","Data":"2ded5a846b5ba7eec9418e0007625a8843c7d352757a30f89d452c590402f5ca"} Jul 06 19:26:32.293554 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.293414 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wk8mp" event={"ID":"7a726848-72dc-4069-965a-d61c7f72a321","Type":"ContainerStarted","Data":"b3bece62121df56e6a91cd1a08966b9716efe2822cda2e5c5eddf75525c33458"} Jul 06 19:26:32.296887 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.296866 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" event={"ID":"84c5d61e-9eee-410f-9591-0e6932cd4a34","Type":"ContainerStarted","Data":"38b2c40ed4b19c9585c69c5653b8f206f3b911ea7d343b0dda4bd17b24863318"} Jul 06 19:26:32.298703 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.298675 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9lclq" event={"ID":"048d362e-572e-45c4-8d5a-ee18eb72e278","Type":"ContainerStarted","Data":"a74cb0814672ffdcd1e366ca3f958b6bc23239c65a2377b79807a1a33ea60a75"} Jul 06 19:26:32.302351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.302316 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cksts" podStartSLOduration=4.059357571 podStartE2EDuration="23.302293625s" totalImagesPullingTime="19.242936054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:31.358587495 +0000 UTC m=+22.733043436" watchObservedRunningTime="2026-07-06 19:26:32.302293625 +0000 UTC m=+23.676749555" Jul 06 19:26:32.345337 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.345283 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wk8mp" podStartSLOduration=4.29583823 podStartE2EDuration="23.345265242s" totalImagesPullingTime="19.049427012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:32.345019474 +0000 UTC m=+23.719475415" watchObservedRunningTime="2026-07-06 19:26:32.345265242 +0000 UTC m=+23.719721213" Jul 06 19:26:32.363759 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.363698 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2pdr9" podStartSLOduration=4.377768116 podStartE2EDuration="23.363677874s" totalImagesPullingTime="18.985909758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:32.363266985 +0000 UTC m=+23.737722928" watchObservedRunningTime="2026-07-06 19:26:32.363677874 +0000 UTC m=+23.738133815" Jul 06 19:26:32.382297 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.382227 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-blvmv" podStartSLOduration=4.351635171 podStartE2EDuration="23.382192406s" totalImagesPullingTime="19.030557235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:32.381947425 +0000 UTC m=+23.756403365" watchObservedRunningTime="2026-07-06 19:26:32.382192406 +0000 UTC m=+23.756648348" Jul 06 19:26:32.409793 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:32.409740 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9lclq" podStartSLOduration=4.338746269 podStartE2EDuration="23.409718149s" totalImagesPullingTime="19.07097188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:32.409671309 +0000 UTC m=+23.784127250" watchObservedRunningTime="2026-07-06 19:26:32.409718149 +0000 UTC m=+23.784174089" Jul 06 19:26:33.097983 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.097371 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 19:26:33.165090 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.164925 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:26:33.097393111Z","UUID":"0ef930a7-08b1-4430-8c04-33b83c81bfbb","Handler":null,"Name":"","Endpoint":""} Jul 06 19:26:33.167988 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.167961 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 19:26:33.168151 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.168012 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 19:26:33.199099 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.199061 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:33.199279 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:33.199221 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:33.199279 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.199260 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:33.199375 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:33.199352 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:33.303860 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.303763 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" event={"ID":"8131de443f47a89b4230bfcbf8311bf6","Type":"ContainerStarted","Data":"4b9bd9ad99de85838387580ea333aaef0b81e3d3d0459ab4939c17db7132cf2d"} Jul 06 19:26:33.309486 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.309447 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" event={"ID":"84c5d61e-9eee-410f-9591-0e6932cd4a34","Type":"ContainerStarted","Data":"48972de73b8a5b08c69300305efb31d7f5dc53cb12e512560740022bc4380ecb"} Jul 06 19:26:33.324126 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:33.324065 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-100.ec2.internal" podStartSLOduration=23.324049126 podStartE2EDuration="23.324049126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:33.323629303 +0000 UTC m=+24.698085244" watchObservedRunningTime="2026-07-06 19:26:33.324049126 +0000 UTC m=+24.698505064" Jul 06 19:26:34.198512 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:34.198434 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:34.198671 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:34.198561 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:34.314852 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:34.314829 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:26:34.315201 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:34.315170 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"422b2845a5092ccbf9bb4fa6f74f3d7bcba4f6ca3aef892c9bf45a93a8fa2cef"} Jul 06 19:26:35.198637 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:35.198606 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:35.198822 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:35.198649 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:35.198822 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:35.198728 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:35.198933 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:35.198882 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:35.320175 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:35.320133 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" event={"ID":"84c5d61e-9eee-410f-9591-0e6932cd4a34","Type":"ContainerStarted","Data":"fb7b07acadd891ed50af29df5b2c29229a41a3bd765890d57583e53b9ed946ca"} Jul 06 19:26:35.340595 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:35.340544 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-plmgw" podStartSLOduration=4.9486239990000005 podStartE2EDuration="26.340523366s" totalImagesPullingTime="21.391899367s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:35.34020786 +0000 UTC m=+26.714663802" watchObservedRunningTime="2026-07-06 19:26:35.340523366 +0000 UTC m=+26.714979306" Jul 06 19:26:36.198027 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:36.197954 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:36.198162 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:36.198068 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:37.047858 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.047667 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:37.048773 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.048757 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:37.198344 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.198308 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:37.198512 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:37.198415 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:37.198512 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.198474 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:37.198582 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:37.198565 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:37.326435 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.326405 2565 generic.go:356] "Generic (PLEG): container finished" podID="51d20679-22ec-4bdf-802a-2b15999befec" containerID="68554aff9779bc771f81b900b86e88843c43185bb0c15540576936ad2989dab8" exitCode=0 Jul 06 19:26:37.326627 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.326491 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf722" event={"ID":"51d20679-22ec-4bdf-802a-2b15999befec","Type":"ContainerDied","Data":"68554aff9779bc771f81b900b86e88843c43185bb0c15540576936ad2989dab8"} Jul 06 19:26:37.330029 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.330013 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:26:37.330409 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.330387 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"1245b7b202f7a1911cead6f4fbbc24ac762a8e9eab9c79b845cf78fdbadbf592"} Jul 06 19:26:37.330714 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.330699 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:37.330901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.330878 2565 scope.go:117] "RemoveContainer" containerID="51b3cfebe9775581edd8987c0b9a6ec99028e282dcdad5d91a9734012826423c" Jul 06 19:26:37.331288 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:37.331267 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wk8mp" Jul 06 19:26:38.198655 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.198621 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:38.199293 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:38.198755 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:38.335898 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.335872 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:26:38.336276 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.336247 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" event={"ID":"805cce93-10de-4cb9-a20e-29561db3c966","Type":"ContainerStarted","Data":"d5f546a12545d26320dbadbf1e576bcf8ab29b380a7b03a1c92f726d8fb7b13c"} Jul 06 19:26:38.336667 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.336644 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:38.336794 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.336715 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:38.336794 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.336783 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:38.355124 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.355098 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:38.355276 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.355222 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:26:38.373520 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.373470 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" podStartSLOduration=9.675225617 podStartE2EDuration="29.3734551s" totalImagesPullingTime="19.698229483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:38.373163247 +0000 UTC m=+29.747619252" watchObservedRunningTime="2026-07-06 19:26:38.3734551 +0000 UTC m=+29.747911040" Jul 06 19:26:38.945626 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.945409 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b6ks2"] Jul 06 19:26:38.945837 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.945752 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:38.945925 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:38.945861 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:38.949514 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.949487 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5lcpf"] Jul 06 19:26:38.949686 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.949593 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:38.949686 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:38.949676 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:38.950015 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.949993 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gndv4"] Jul 06 19:26:38.950098 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:38.950087 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:38.950193 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:38.950176 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:39.340339 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:39.340303 2565 generic.go:356] "Generic (PLEG): container finished" podID="51d20679-22ec-4bdf-802a-2b15999befec" containerID="d979b617b3d8546676980a68158dc46d0838da307ab35aaee5f9e90771212537" exitCode=0 Jul 06 19:26:39.340702 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:39.340381 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf722" event={"ID":"51d20679-22ec-4bdf-802a-2b15999befec","Type":"ContainerDied","Data":"d979b617b3d8546676980a68158dc46d0838da307ab35aaee5f9e90771212537"} Jul 06 19:26:41.198439 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:41.198404 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:41.198903 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:41.198404 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:41.198903 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:41.198526 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:41.198903 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:41.198404 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:41.198903 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:41.198600 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:41.198903 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:41.198651 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:41.347741 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:41.347703 2565 generic.go:356] "Generic (PLEG): container finished" podID="51d20679-22ec-4bdf-802a-2b15999befec" containerID="d046840c05826691f9a4af5dfeca4cca9cfd151b4cb33535fb5bc32f7b92545b" exitCode=0 Jul 06 19:26:41.347896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:41.347763 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf722" event={"ID":"51d20679-22ec-4bdf-802a-2b15999befec","Type":"ContainerDied","Data":"d046840c05826691f9a4af5dfeca4cca9cfd151b4cb33535fb5bc32f7b92545b"} Jul 06 19:26:42.927072 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:42.927032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:42.927072 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:42.927074 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:42.927582 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:42.927180 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:42.927582 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:42.927250 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret podName:17bca18e-95ba-4d4b-b844-7f7b52c769f2 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:14.927234433 +0000 UTC m=+66.301690351 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret") pod "global-pull-secret-syncer-b6ks2" (UID: "17bca18e-95ba-4d4b-b844-7f7b52c769f2") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:26:42.927582 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:42.927182 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:42.927582 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:42.927320 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:27:14.927301727 +0000 UTC m=+66.301757657 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:26:43.027835 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.027803 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:43.028022 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.027985 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:26:43.028022 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.028012 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:26:43.028022 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.028022 2565 projected.go:196] Error preparing data for projected volume kube-api-access-dzq8t for pod openshift-network-diagnostics/network-check-target-gndv4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:43.028129 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.028083 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t podName:e7cbcf2f-c466-4465-ad91-5094d5bb4cb9 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:15.028069202 +0000 UTC m=+66.402525132 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dzq8t" (UniqueName: "kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t") pod "network-check-target-gndv4" (UID: "e7cbcf2f-c466-4465-ad91-5094d5bb4cb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:26:43.198855 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.198765 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:43.198855 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.198810 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:43.199120 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.198906 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6ks2" podUID="17bca18e-95ba-4d4b-b844-7f7b52c769f2" Jul 06 19:26:43.199120 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.198765 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:43.199249 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.199186 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gndv4" podUID="e7cbcf2f-c466-4465-ad91-5094d5bb4cb9" Jul 06 19:26:43.199383 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.199359 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:26:43.410143 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.410110 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-100.ec2.internal" event="NodeReady" Jul 06 19:26:43.410365 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.410297 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:26:43.467979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.467895 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gggzl"] Jul 06 19:26:43.482558 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.482516 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gflxp"] Jul 06 19:26:43.482810 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.482784 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.485668 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.485647 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7f4qs\"" Jul 06 19:26:43.486117 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.486095 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:26:43.486664 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.486645 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:26:43.495151 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.495068 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gggzl"] Jul 06 19:26:43.495151 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.495120 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gflxp"] Jul 06 19:26:43.495151 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.495150 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gflxp" Jul 06 19:26:43.498111 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.497949 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:26:43.498111 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.497965 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:26:43.498111 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.498003 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kkk98\"" Jul 06 19:26:43.498111 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.498010 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:26:43.633020 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.632777 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efbb99a-2d00-47b8-9188-4a35489cff63-tmp-dir\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.633020 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.633024 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zkln\" (UniqueName: \"kubernetes.io/projected/9efbb99a-2d00-47b8-9188-4a35489cff63-kube-api-access-7zkln\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.633307 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.633060 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d93137-2531-41a7-9523-a824b57b3dae-cert\") pod \"ingress-canary-gflxp\" (UID: \"a7d93137-2531-41a7-9523-a824b57b3dae\") " pod="openshift-ingress-canary/ingress-canary-gflxp" Jul 06 19:26:43.633307 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.633134 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9efbb99a-2d00-47b8-9188-4a35489cff63-config-volume\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.633307 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.633149 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgh8d\" (UniqueName: \"kubernetes.io/projected/a7d93137-2531-41a7-9523-a824b57b3dae-kube-api-access-dgh8d\") pod \"ingress-canary-gflxp\" (UID: \"a7d93137-2531-41a7-9523-a824b57b3dae\") " pod="openshift-ingress-canary/ingress-canary-gflxp" Jul 06 19:26:43.633307 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.633180 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.734179 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.734106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efbb99a-2d00-47b8-9188-4a35489cff63-tmp-dir\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.734179 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.734138 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zkln\" (UniqueName: \"kubernetes.io/projected/9efbb99a-2d00-47b8-9188-4a35489cff63-kube-api-access-7zkln\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.734179 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.734161 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d93137-2531-41a7-9523-a824b57b3dae-cert\") pod \"ingress-canary-gflxp\" (UID: \"a7d93137-2531-41a7-9523-a824b57b3dae\") " pod="openshift-ingress-canary/ingress-canary-gflxp" Jul 06 19:26:43.734525 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.734316 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9efbb99a-2d00-47b8-9188-4a35489cff63-config-volume\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.734525 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.734341 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgh8d\" (UniqueName: \"kubernetes.io/projected/a7d93137-2531-41a7-9523-a824b57b3dae-kube-api-access-dgh8d\") pod \"ingress-canary-gflxp\" (UID: \"a7d93137-2531-41a7-9523-a824b57b3dae\") " pod="openshift-ingress-canary/ingress-canary-gflxp" Jul 06 19:26:43.734525 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.734380 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.734525 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.734477 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:26:43.734741 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:43.734534 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls podName:9efbb99a-2d00-47b8-9188-4a35489cff63 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:44.234514442 +0000 UTC m=+35.608970361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls") pod "dns-default-gggzl" (UID: "9efbb99a-2d00-47b8-9188-4a35489cff63") : secret "dns-default-metrics-tls" not found Jul 06 19:26:43.734938 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.734879 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efbb99a-2d00-47b8-9188-4a35489cff63-tmp-dir\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.735287 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.735266 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9efbb99a-2d00-47b8-9188-4a35489cff63-config-volume\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.739654 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.739628 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d93137-2531-41a7-9523-a824b57b3dae-cert\") pod \"ingress-canary-gflxp\" (UID: \"a7d93137-2531-41a7-9523-a824b57b3dae\") " pod="openshift-ingress-canary/ingress-canary-gflxp" Jul 06 19:26:43.745867 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.745832 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgh8d\" (UniqueName: \"kubernetes.io/projected/a7d93137-2531-41a7-9523-a824b57b3dae-kube-api-access-dgh8d\") pod \"ingress-canary-gflxp\" (UID: \"a7d93137-2531-41a7-9523-a824b57b3dae\") " pod="openshift-ingress-canary/ingress-canary-gflxp" Jul 06 19:26:43.745867 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.745857 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zkln\" (UniqueName: \"kubernetes.io/projected/9efbb99a-2d00-47b8-9188-4a35489cff63-kube-api-access-7zkln\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:43.805078 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:43.805040 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gflxp" Jul 06 19:26:44.016350 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:44.016269 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gflxp"] Jul 06 19:26:44.021355 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:26:44.021326 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d93137_2531_41a7_9523_a824b57b3dae.slice/crio-0781844a9e6c245c25aad05224358818d423738ffacaf0eb15dfa49ed8b09ce8 WatchSource:0}: Error finding container 0781844a9e6c245c25aad05224358818d423738ffacaf0eb15dfa49ed8b09ce8: Status 404 returned error can't find the container with id 0781844a9e6c245c25aad05224358818d423738ffacaf0eb15dfa49ed8b09ce8 Jul 06 19:26:44.238582 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:44.238538 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:44.238865 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:44.238675 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:26:44.238865 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:44.238805 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls podName:9efbb99a-2d00-47b8-9188-4a35489cff63 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:45.238783865 +0000 UTC m=+36.613239800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls") pod "dns-default-gggzl" (UID: "9efbb99a-2d00-47b8-9188-4a35489cff63") : secret "dns-default-metrics-tls" not found Jul 06 19:26:44.356426 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:44.356395 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gflxp" event={"ID":"a7d93137-2531-41a7-9523-a824b57b3dae","Type":"ContainerStarted","Data":"0781844a9e6c245c25aad05224358818d423738ffacaf0eb15dfa49ed8b09ce8"} Jul 06 19:26:45.199059 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.199020 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:26:45.199700 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.199183 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:26:45.199700 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.199020 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:26:45.203890 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.203830 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:26:45.203890 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.203864 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cdwtt\"" Jul 06 19:26:45.203890 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.203881 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t5qdm\"" Jul 06 19:26:45.204140 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.203895 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:26:45.204140 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.203830 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:26:45.204140 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.203868 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:26:45.247074 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:45.247037 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:45.247254 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:45.247181 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:26:45.247308 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:45.247282 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls podName:9efbb99a-2d00-47b8-9188-4a35489cff63 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:47.247259029 +0000 UTC m=+38.621714950 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls") pod "dns-default-gggzl" (UID: "9efbb99a-2d00-47b8-9188-4a35489cff63") : secret "dns-default-metrics-tls" not found Jul 06 19:26:47.264160 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:47.263951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:47.264586 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:47.264131 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:26:47.264586 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:47.264306 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls podName:9efbb99a-2d00-47b8-9188-4a35489cff63 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:51.264284265 +0000 UTC m=+42.638740185 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls") pod "dns-default-gggzl" (UID: "9efbb99a-2d00-47b8-9188-4a35489cff63") : secret "dns-default-metrics-tls" not found Jul 06 19:26:47.366933 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:47.366899 2565 generic.go:356] "Generic (PLEG): container finished" podID="51d20679-22ec-4bdf-802a-2b15999befec" containerID="8661e0a5a9f94eea6337e070df8fa824d45f9923bd5fe5e399297b07c173506b" exitCode=0 Jul 06 19:26:47.367102 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:47.366961 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf722" event={"ID":"51d20679-22ec-4bdf-802a-2b15999befec","Type":"ContainerDied","Data":"8661e0a5a9f94eea6337e070df8fa824d45f9923bd5fe5e399297b07c173506b"} Jul 06 19:26:48.374371 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:48.374336 2565 generic.go:356] "Generic (PLEG): container finished" podID="51d20679-22ec-4bdf-802a-2b15999befec" containerID="0a6ed562b6bfd1192d95a01340c609f624ab34c24f5aa4cfaab271fc1354ddf6" exitCode=0 Jul 06 19:26:48.374812 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:48.374416 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf722" event={"ID":"51d20679-22ec-4bdf-802a-2b15999befec","Type":"ContainerDied","Data":"0a6ed562b6bfd1192d95a01340c609f624ab34c24f5aa4cfaab271fc1354ddf6"} Jul 06 19:26:48.375892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:48.375865 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gflxp" event={"ID":"a7d93137-2531-41a7-9523-a824b57b3dae","Type":"ContainerStarted","Data":"01e69e0b7db29d101937613deb3a331b83df47afe9bc68f257cb9d01cacba788"} Jul 06 19:26:48.424816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:48.424755 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gflxp" podStartSLOduration=1.512129681 podStartE2EDuration="5.424730497s" totalImagesPullingTime="3.912600816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:48.423971141 +0000 UTC m=+39.798427079" watchObservedRunningTime="2026-07-06 19:26:48.424730497 +0000 UTC m=+39.799186437" Jul 06 19:26:49.381124 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:49.381092 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf722" event={"ID":"51d20679-22ec-4bdf-802a-2b15999befec","Type":"ContainerStarted","Data":"de9523fc97286233425a8f1012a6a362416fa38fbe6b537ea6b2ef5825c8949a"} Jul 06 19:26:49.414296 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:49.414202 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wf722" podStartSLOduration=9.675738232 podStartE2EDuration="40.414181756s" totalImagesPullingTime="30.738443524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:49.413888731 +0000 UTC m=+40.788344669" watchObservedRunningTime="2026-07-06 19:26:49.414181756 +0000 UTC m=+40.788637697" Jul 06 19:26:51.292564 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:51.292511 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:51.292952 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:51.292650 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:26:51.292952 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:51.292710 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls podName:9efbb99a-2d00-47b8-9188-4a35489cff63 nodeName:}" failed. No retries permitted until 2026-07-06 19:26:59.292695031 +0000 UTC m=+50.667150949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls") pod "dns-default-gggzl" (UID: "9efbb99a-2d00-47b8-9188-4a35489cff63") : secret "dns-default-metrics-tls" not found Jul 06 19:26:59.347886 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:26:59.347836 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:26:59.348342 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:59.348008 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:26:59.348342 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:26:59.348084 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls podName:9efbb99a-2d00-47b8-9188-4a35489cff63 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:15.348063994 +0000 UTC m=+66.722519930 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls") pod "dns-default-gggzl" (UID: "9efbb99a-2d00-47b8-9188-4a35489cff63") : secret "dns-default-metrics-tls" not found Jul 06 19:27:10.357357 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:10.357329 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ls98" Jul 06 19:27:14.947692 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:14.947653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:27:14.947692 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:14.947692 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:27:14.950942 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:14.950921 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:27:14.951074 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:14.951060 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:27:14.958880 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:27:14.958858 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 19:27:14.958943 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:27:14.958932 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:28:18.958914059 +0000 UTC m=+130.333369977 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : secret "metrics-daemon-secret" not found Jul 06 19:27:14.961934 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:14.961905 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17bca18e-95ba-4d4b-b844-7f7b52c769f2-original-pull-secret\") pod \"global-pull-secret-syncer-b6ks2\" (UID: \"17bca18e-95ba-4d4b-b844-7f7b52c769f2\") " pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:27:15.048200 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.048162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:27:15.051651 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.051630 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:27:15.062276 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.062240 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:27:15.072063 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.072036 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzq8t\" (UniqueName: \"kubernetes.io/projected/e7cbcf2f-c466-4465-ad91-5094d5bb4cb9-kube-api-access-dzq8t\") pod \"network-check-target-gndv4\" (UID: \"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9\") " pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:27:15.214909 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.214829 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cdwtt\"" Jul 06 19:27:15.218713 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.218696 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6ks2" Jul 06 19:27:15.223523 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.223481 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:27:15.350851 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.350821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:27:15.351001 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:27:15.350933 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:27:15.351001 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:27:15.350987 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls podName:9efbb99a-2d00-47b8-9188-4a35489cff63 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:47.350971438 +0000 UTC m=+98.725427357 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls") pod "dns-default-gggzl" (UID: "9efbb99a-2d00-47b8-9188-4a35489cff63") : secret "dns-default-metrics-tls" not found Jul 06 19:27:15.354418 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.354384 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gndv4"] Jul 06 19:27:15.358195 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:27:15.358170 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7cbcf2f_c466_4465_ad91_5094d5bb4cb9.slice/crio-bd32524a656eba97529f18b120bca62699dd1da22843e40828a44e8004fe5ad7 WatchSource:0}: Error finding container bd32524a656eba97529f18b120bca62699dd1da22843e40828a44e8004fe5ad7: Status 404 returned error can't find the container with id bd32524a656eba97529f18b120bca62699dd1da22843e40828a44e8004fe5ad7 Jul 06 19:27:15.367487 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.367450 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b6ks2"] Jul 06 19:27:15.370613 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:27:15.370585 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17bca18e_95ba_4d4b_b844_7f7b52c769f2.slice/crio-43d31655498843004d8020960bfaa59fe93b0e564b67f1fb8baabff9ba1a6a6d WatchSource:0}: Error finding container 43d31655498843004d8020960bfaa59fe93b0e564b67f1fb8baabff9ba1a6a6d: Status 404 returned error can't find the container with id 43d31655498843004d8020960bfaa59fe93b0e564b67f1fb8baabff9ba1a6a6d Jul 06 19:27:15.445584 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.445544 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gndv4" event={"ID":"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9","Type":"ContainerStarted","Data":"bd32524a656eba97529f18b120bca62699dd1da22843e40828a44e8004fe5ad7"} Jul 06 19:27:15.446690 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:15.446663 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b6ks2" event={"ID":"17bca18e-95ba-4d4b-b844-7f7b52c769f2","Type":"ContainerStarted","Data":"43d31655498843004d8020960bfaa59fe93b0e564b67f1fb8baabff9ba1a6a6d"} Jul 06 19:27:20.468236 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:20.468177 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gndv4" event={"ID":"e7cbcf2f-c466-4465-ad91-5094d5bb4cb9","Type":"ContainerStarted","Data":"f22f6be725145bb7ee56ee71864a374d846053fd97e8f8cb4d8f5702d829b79f"} Jul 06 19:27:20.468719 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:20.468297 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:27:20.469713 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:20.469685 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b6ks2" event={"ID":"17bca18e-95ba-4d4b-b844-7f7b52c769f2","Type":"ContainerStarted","Data":"fa0162670aeb369be3504d4fee252c02fcb46e0008309577e81714132d0317c4"} Jul 06 19:27:20.488066 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:20.488018 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gndv4" podStartSLOduration=66.822092226 podStartE2EDuration="1m11.488005974s" totalImagesPullingTime="4.665913748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:20.486252348 +0000 UTC m=+71.860708285" watchObservedRunningTime="2026-07-06 19:27:20.488005974 +0000 UTC m=+71.862461913" Jul 06 19:27:20.504114 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:20.504067 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b6ks2" podStartSLOduration=66.847900226 podStartE2EDuration="1m11.504053521s" totalImagesPullingTime="4.656153295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:20.503958842 +0000 UTC m=+71.878414783" watchObservedRunningTime="2026-07-06 19:27:20.504053521 +0000 UTC m=+71.878509461" Jul 06 19:27:47.364677 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:47.364625 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:27:47.365335 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:27:47.364800 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:27:47.365335 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:27:47.364881 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls podName:9efbb99a-2d00-47b8-9188-4a35489cff63 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:51.364859248 +0000 UTC m=+162.739315169 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls") pod "dns-default-gggzl" (UID: "9efbb99a-2d00-47b8-9188-4a35489cff63") : secret "dns-default-metrics-tls" not found Jul 06 19:27:51.475486 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:27:51.475456 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gndv4" Jul 06 19:28:12.810506 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.810472 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w"] Jul 06 19:28:12.812146 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.812131 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:12.815443 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.815421 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 19:28:12.816072 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.816057 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:28:12.821523 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.819428 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:28:12.821523 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.819770 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cxfk5\"" Jul 06 19:28:12.821523 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.821017 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 19:28:12.827880 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.827859 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w"] Jul 06 19:28:12.941266 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.941227 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:12.941441 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.941273 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:12.941441 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:12.941350 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47fm7\" (UniqueName: \"kubernetes.io/projected/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-kube-api-access-47fm7\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:13.042430 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:13.042384 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47fm7\" (UniqueName: \"kubernetes.io/projected/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-kube-api-access-47fm7\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:13.042601 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:13.042441 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:13.042601 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:13.042462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:13.042601 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:13.042556 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:13.042698 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:13.042615 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls podName:cf9471e6-bb11-4bdf-b34e-f2821b1551a7 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:13.542597291 +0000 UTC m=+124.917053226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nct7w" (UID: "cf9471e6-bb11-4bdf-b34e-f2821b1551a7") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:13.043126 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:13.043106 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:13.052174 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:13.052142 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47fm7\" (UniqueName: \"kubernetes.io/projected/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-kube-api-access-47fm7\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:13.545815 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:13.545765 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:13.546012 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:13.545932 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:13.546012 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:13.545997 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls podName:cf9471e6-bb11-4bdf-b34e-f2821b1551a7 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:14.545983094 +0000 UTC m=+125.920439012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nct7w" (UID: "cf9471e6-bb11-4bdf-b34e-f2821b1551a7") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:14.553096 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:14.553058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:14.553598 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:14.553178 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:14.553598 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:14.553275 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls podName:cf9471e6-bb11-4bdf-b34e-f2821b1551a7 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:16.5532525 +0000 UTC m=+127.927708439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nct7w" (UID: "cf9471e6-bb11-4bdf-b34e-f2821b1551a7") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:16.476129 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.476094 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hbh7b"] Jul 06 19:28:16.478269 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.478253 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.483694 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:16.483658 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"insights-runtime-extractor-hbh7b\" is forbidden: User \"system:node:ip-10-0-143-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-143-100.ec2.internal' and this object" podUID="c3d34406-18c8-4adf-ba4b-4ed16cb0e367" pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.483800 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:16.483726 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-143-100.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-143-100.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 06 19:28:16.483800 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:16.483728 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-143-100.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-143-100.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" Jul 06 19:28:16.484561 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:16.484407 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-143-100.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-143-100.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 19:28:16.487417 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:16.487393 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-sa-dockercfg-k82dj\" is forbidden: User \"system:node:ip-10-0-143-100.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-143-100.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k82dj\"" type="*v1.Secret" Jul 06 19:28:16.498957 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:16.498931 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-tls\" is forbidden: User \"system:node:ip-10-0-143-100.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-143-100.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" Jul 06 19:28:16.503908 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.503884 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hbh7b"] Jul 06 19:28:16.567395 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.567354 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-data-volume\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.567558 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.567413 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:16.567558 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.567464 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-crio-socket\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.567558 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.567494 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.567558 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:16.567524 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:16.567711 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.567524 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9m46\" (UniqueName: \"kubernetes.io/projected/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-api-access-v9m46\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.567711 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:16.567575 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls podName:cf9471e6-bb11-4bdf-b34e-f2821b1551a7 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:20.567560357 +0000 UTC m=+131.942016291 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nct7w" (UID: "cf9471e6-bb11-4bdf-b34e-f2821b1551a7") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:16.567711 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.567624 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.668293 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.668251 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-crio-socket\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.668293 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.668292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.668529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.668316 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9m46\" (UniqueName: \"kubernetes.io/projected/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-api-access-v9m46\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.668529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.668341 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.668529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.668381 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-crio-socket\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.668529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.668432 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-data-volume\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.668753 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.668736 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-data-volume\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:16.707969 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.707934 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6zvvd"] Jul 06 19:28:16.709986 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.709971 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.719668 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.719646 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 19:28:16.719668 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.719659 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:28:16.721227 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.721192 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 19:28:16.721329 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.721268 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 19:28:16.721329 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.721274 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-575jc\"" Jul 06 19:28:16.725923 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.725900 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 19:28:16.729168 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.729114 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6zvvd"] Jul 06 19:28:16.869584 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.869525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nstqk\" (UniqueName: \"kubernetes.io/projected/6f168e98-d917-4b91-9dfd-1a47192a428d-kube-api-access-nstqk\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.869584 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.869598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f168e98-d917-4b91-9dfd-1a47192a428d-serving-cert\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.869843 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.869617 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f168e98-d917-4b91-9dfd-1a47192a428d-trusted-ca\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.869843 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.869671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f168e98-d917-4b91-9dfd-1a47192a428d-config\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.970560 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.970508 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nstqk\" (UniqueName: \"kubernetes.io/projected/6f168e98-d917-4b91-9dfd-1a47192a428d-kube-api-access-nstqk\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.970755 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.970597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f168e98-d917-4b91-9dfd-1a47192a428d-serving-cert\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.970755 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.970620 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f168e98-d917-4b91-9dfd-1a47192a428d-trusted-ca\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.970755 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.970646 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f168e98-d917-4b91-9dfd-1a47192a428d-config\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.971341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.971316 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f168e98-d917-4b91-9dfd-1a47192a428d-config\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.971524 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.971502 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f168e98-d917-4b91-9dfd-1a47192a428d-trusted-ca\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.972849 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.972820 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f168e98-d917-4b91-9dfd-1a47192a428d-serving-cert\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:16.981338 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:16.981286 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nstqk\" (UniqueName: \"kubernetes.io/projected/6f168e98-d917-4b91-9dfd-1a47192a428d-kube-api-access-nstqk\") pod \"console-operator-68b757fb67-6zvvd\" (UID: \"6f168e98-d917-4b91-9dfd-1a47192a428d\") " pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:17.017790 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:17.017754 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:17.148654 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:17.148625 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-6zvvd"] Jul 06 19:28:17.151850 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:17.151813 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f168e98_d917_4b91_9dfd_1a47192a428d.slice/crio-f95f73ff0065a89b1119584c250dfcc23d95db31827e55103b0b21cc7a86a87a WatchSource:0}: Error finding container f95f73ff0065a89b1119584c250dfcc23d95db31827e55103b0b21cc7a86a87a: Status 404 returned error can't find the container with id f95f73ff0065a89b1119584c250dfcc23d95db31827e55103b0b21cc7a86a87a Jul 06 19:28:17.608260 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:17.608226 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" event={"ID":"6f168e98-d917-4b91-9dfd-1a47192a428d","Type":"ContainerStarted","Data":"f95f73ff0065a89b1119584c250dfcc23d95db31827e55103b0b21cc7a86a87a"} Jul 06 19:28:17.619399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:17.619368 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:28:17.668638 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:17.668603 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: failed to sync configmap cache: timed out waiting for the condition Jul 06 19:28:17.668800 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:17.668685 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:18.168667419 +0000 UTC m=+129.543123337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : failed to sync configmap cache: timed out waiting for the condition Jul 06 19:28:17.668800 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:17.668602 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: failed to sync secret cache: timed out waiting for the condition Jul 06 19:28:17.668800 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:17.668764 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:18.168745952 +0000 UTC m=+129.543201871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:28:17.859650 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:17.859576 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:28:17.904943 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:17.904901 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:28:18.045584 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:18.045549 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k82dj\"" Jul 06 19:28:18.066456 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:18.066423 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:28:18.070105 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:18.070072 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9m46\" (UniqueName: \"kubernetes.io/projected/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-api-access-v9m46\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:18.181622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:18.181530 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:18.181622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:18.181579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:18.181838 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:18.181686 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:28:18.181838 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:18.181749 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:19.18173398 +0000 UTC m=+130.556189898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : secret "insights-runtime-extractor-tls" not found Jul 06 19:28:18.181838 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:18.181689 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:28:18.181838 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:18.181839 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:19.181821471 +0000 UTC m=+130.556277410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : configmap "kube-rbac-proxy" not found Jul 06 19:28:18.442180 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:18.442105 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-blvmv_d1df10a1-d344-41da-a87f-1df534ffc631/dns-node-resolver/0.log" Jul 06 19:28:18.987553 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:18.987511 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:28:18.988053 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:18.987676 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 19:28:18.988053 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:18.987760 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs podName:14de08b3-3e7b-4641-97c8-7140cb4c24ec nodeName:}" failed. No retries permitted until 2026-07-06 19:30:20.98773815 +0000 UTC m=+252.362194074 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs") pod "network-metrics-daemon-5lcpf" (UID: "14de08b3-3e7b-4641-97c8-7140cb4c24ec") : secret "metrics-daemon-secret" not found Jul 06 19:28:19.043450 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:19.043415 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pdr9_77d302df-429b-47c4-a97c-c8097c1db9fd/node-ca/0.log" Jul 06 19:28:19.189174 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:19.189133 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:19.189369 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:19.189237 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:28:19.189369 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:19.189303 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:19.189369 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:19.189359 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:21.189338821 +0000 UTC m=+132.563794744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : configmap "kube-rbac-proxy" not found Jul 06 19:28:19.189558 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:19.189451 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:28:19.189558 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:19.189514 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:21.189493793 +0000 UTC m=+132.563949719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : secret "insights-runtime-extractor-tls" not found Jul 06 19:28:19.616514 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:19.616483 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/0.log" Jul 06 19:28:19.616709 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:19.616528 2565 generic.go:356] "Generic (PLEG): container finished" podID="6f168e98-d917-4b91-9dfd-1a47192a428d" containerID="ff216724402a4fb99c5e3f462a77db8a089607b95d91501c3f14242b2598ca9c" exitCode=255 Jul 06 19:28:19.616709 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:19.616597 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" event={"ID":"6f168e98-d917-4b91-9dfd-1a47192a428d","Type":"ContainerDied","Data":"ff216724402a4fb99c5e3f462a77db8a089607b95d91501c3f14242b2598ca9c"} Jul 06 19:28:19.616981 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:19.616856 2565 scope.go:117] "RemoveContainer" containerID="ff216724402a4fb99c5e3f462a77db8a089607b95d91501c3f14242b2598ca9c" Jul 06 19:28:20.244073 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.244043 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gflxp_a7d93137-2531-41a7-9523-a824b57b3dae/serve-healthcheck-canary/0.log" Jul 06 19:28:20.600838 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.600797 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:20.600975 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:20.600915 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:20.600975 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:20.600972 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls podName:cf9471e6-bb11-4bdf-b34e-f2821b1551a7 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:28.600957858 +0000 UTC m=+139.975413776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nct7w" (UID: "cf9471e6-bb11-4bdf-b34e-f2821b1551a7") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:20.621130 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.621100 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:28:20.621497 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.621482 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/0.log" Jul 06 19:28:20.621548 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.621515 2565 generic.go:356] "Generic (PLEG): container finished" podID="6f168e98-d917-4b91-9dfd-1a47192a428d" containerID="c23f91e747b00c644cca3d041d7d31c3ed77905a5667aacb9c4d37f7fa5a7e57" exitCode=255 Jul 06 19:28:20.621579 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.621546 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" event={"ID":"6f168e98-d917-4b91-9dfd-1a47192a428d","Type":"ContainerDied","Data":"c23f91e747b00c644cca3d041d7d31c3ed77905a5667aacb9c4d37f7fa5a7e57"} Jul 06 19:28:20.621623 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.621612 2565 scope.go:117] "RemoveContainer" containerID="ff216724402a4fb99c5e3f462a77db8a089607b95d91501c3f14242b2598ca9c" Jul 06 19:28:20.621846 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.621826 2565 scope.go:117] "RemoveContainer" containerID="c23f91e747b00c644cca3d041d7d31c3ed77905a5667aacb9c4d37f7fa5a7e57" Jul 06 19:28:20.622034 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:20.622015 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6zvvd_openshift-console-operator(6f168e98-d917-4b91-9dfd-1a47192a428d)\"" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" podUID="6f168e98-d917-4b91-9dfd-1a47192a428d" Jul 06 19:28:20.758138 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.758103 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g"] Jul 06 19:28:20.760598 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.760575 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:20.763760 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.763731 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:28:20.763869 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.763777 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qw862\"" Jul 06 19:28:20.763869 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.763812 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 19:28:20.763963 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.763932 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 19:28:20.764139 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.764121 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 19:28:20.774594 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.774576 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g"] Jul 06 19:28:20.902825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.902728 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/543e3ca8-22d4-4b02-989f-af18444f9f57-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:20.902825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.902790 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/543e3ca8-22d4-4b02-989f-af18444f9f57-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:20.903056 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:20.902859 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rglqq\" (UniqueName: \"kubernetes.io/projected/543e3ca8-22d4-4b02-989f-af18444f9f57-kube-api-access-rglqq\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:21.003718 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.003683 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/543e3ca8-22d4-4b02-989f-af18444f9f57-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:21.003816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.003732 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rglqq\" (UniqueName: \"kubernetes.io/projected/543e3ca8-22d4-4b02-989f-af18444f9f57-kube-api-access-rglqq\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:21.003816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.003809 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/543e3ca8-22d4-4b02-989f-af18444f9f57-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:21.004371 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.004343 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/543e3ca8-22d4-4b02-989f-af18444f9f57-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:21.006008 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.005986 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/543e3ca8-22d4-4b02-989f-af18444f9f57-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:21.013569 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.013549 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rglqq\" (UniqueName: \"kubernetes.io/projected/543e3ca8-22d4-4b02-989f-af18444f9f57-kube-api-access-rglqq\") pod \"kube-storage-version-migrator-operator-846bf668f7-hrn4g\" (UID: \"543e3ca8-22d4-4b02-989f-af18444f9f57\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:21.069753 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.069714 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" Jul 06 19:28:21.194304 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.194225 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g"] Jul 06 19:28:21.197351 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:21.197325 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543e3ca8_22d4_4b02_989f_af18444f9f57.slice/crio-6b8a5fe5b356bfede40c65b556a5bdb11e8dd4c44618c69d970e933c3fb76136 WatchSource:0}: Error finding container 6b8a5fe5b356bfede40c65b556a5bdb11e8dd4c44618c69d970e933c3fb76136: Status 404 returned error can't find the container with id 6b8a5fe5b356bfede40c65b556a5bdb11e8dd4c44618c69d970e933c3fb76136 Jul 06 19:28:21.206091 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.206066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:21.206261 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.206100 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:21.206261 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:21.206170 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:28:21.206261 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:21.206234 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:25.206203725 +0000 UTC m=+136.580659644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : configmap "kube-rbac-proxy" not found Jul 06 19:28:21.206261 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:21.206249 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:28:21.206446 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:21.206310 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:25.206292615 +0000 UTC m=+136.580748547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : secret "insights-runtime-extractor-tls" not found Jul 06 19:28:21.625356 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.625323 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:28:21.625780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.625666 2565 scope.go:117] "RemoveContainer" containerID="c23f91e747b00c644cca3d041d7d31c3ed77905a5667aacb9c4d37f7fa5a7e57" Jul 06 19:28:21.625851 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:21.625828 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6zvvd_openshift-console-operator(6f168e98-d917-4b91-9dfd-1a47192a428d)\"" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" podUID="6f168e98-d917-4b91-9dfd-1a47192a428d" Jul 06 19:28:21.626545 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:21.626523 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" event={"ID":"543e3ca8-22d4-4b02-989f-af18444f9f57","Type":"ContainerStarted","Data":"6b8a5fe5b356bfede40c65b556a5bdb11e8dd4c44618c69d970e933c3fb76136"} Jul 06 19:28:22.647154 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.647123 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7"] Jul 06 19:28:22.648935 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.648917 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.652156 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.652128 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 19:28:22.653324 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.653306 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:28:22.653423 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.653324 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 19:28:22.653423 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.653327 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9fkjc\"" Jul 06 19:28:22.653423 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.653362 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 19:28:22.661923 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.661900 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7"] Jul 06 19:28:22.819323 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.819281 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl4l9\" (UniqueName: \"kubernetes.io/projected/322e1481-512e-4f40-abcd-f12866faa4f4-kube-api-access-pl4l9\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.819515 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.819451 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/322e1481-512e-4f40-abcd-f12866faa4f4-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.819515 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.819475 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/322e1481-512e-4f40-abcd-f12866faa4f4-config\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.920727 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.920631 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/322e1481-512e-4f40-abcd-f12866faa4f4-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.920727 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.920677 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/322e1481-512e-4f40-abcd-f12866faa4f4-config\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.920964 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.920728 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl4l9\" (UniqueName: \"kubernetes.io/projected/322e1481-512e-4f40-abcd-f12866faa4f4-kube-api-access-pl4l9\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.921442 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.921412 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/322e1481-512e-4f40-abcd-f12866faa4f4-config\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.923228 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.923190 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/322e1481-512e-4f40-abcd-f12866faa4f4-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.931541 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.931516 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl4l9\" (UniqueName: \"kubernetes.io/projected/322e1481-512e-4f40-abcd-f12866faa4f4-kube-api-access-pl4l9\") pod \"service-ca-operator-5d46d9ffd4-skfd7\" (UID: \"322e1481-512e-4f40-abcd-f12866faa4f4\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:22.957596 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:22.957558 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" Jul 06 19:28:23.347636 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.347594 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7"] Jul 06 19:28:23.353106 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:23.353073 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322e1481_512e_4f40_abcd_f12866faa4f4.slice/crio-a23dcdc9ec0eaab20bc2b096b2ffbbd2dc2d81a099db17847a714184522ae7c5 WatchSource:0}: Error finding container a23dcdc9ec0eaab20bc2b096b2ffbbd2dc2d81a099db17847a714184522ae7c5: Status 404 returned error can't find the container with id a23dcdc9ec0eaab20bc2b096b2ffbbd2dc2d81a099db17847a714184522ae7c5 Jul 06 19:28:23.472777 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.472685 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj"] Jul 06 19:28:23.474706 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.474688 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj" Jul 06 19:28:23.477529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.477510 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qh6wz\"" Jul 06 19:28:23.493611 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.493577 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj"] Jul 06 19:28:23.627499 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.627463 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhthk\" (UniqueName: \"kubernetes.io/projected/47db4515-9fde-435f-a631-fe81c9c15e5c-kube-api-access-lhthk\") pod \"network-check-source-989f8b8dc-d5shj\" (UID: \"47db4515-9fde-435f-a631-fe81c9c15e5c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj" Jul 06 19:28:23.632933 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.632898 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" event={"ID":"322e1481-512e-4f40-abcd-f12866faa4f4","Type":"ContainerStarted","Data":"a23dcdc9ec0eaab20bc2b096b2ffbbd2dc2d81a099db17847a714184522ae7c5"} Jul 06 19:28:23.634351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.634318 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" event={"ID":"543e3ca8-22d4-4b02-989f-af18444f9f57","Type":"ContainerStarted","Data":"585b35edc544b83144d905a92672b24a9e1e35ae0dffed9e43c3dae3a9fa2566"} Jul 06 19:28:23.658522 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.658468 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" podStartSLOduration=1.612063648 podStartE2EDuration="3.658453213s" totalImagesPullingTime="2.046389565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:23.657670689 +0000 UTC m=+135.032126628" watchObservedRunningTime="2026-07-06 19:28:23.658453213 +0000 UTC m=+135.032909152" Jul 06 19:28:23.728403 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.728318 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhthk\" (UniqueName: \"kubernetes.io/projected/47db4515-9fde-435f-a631-fe81c9c15e5c-kube-api-access-lhthk\") pod \"network-check-source-989f8b8dc-d5shj\" (UID: \"47db4515-9fde-435f-a631-fe81c9c15e5c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj" Jul 06 19:28:23.741417 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.741392 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhthk\" (UniqueName: \"kubernetes.io/projected/47db4515-9fde-435f-a631-fe81c9c15e5c-kube-api-access-lhthk\") pod \"network-check-source-989f8b8dc-d5shj\" (UID: \"47db4515-9fde-435f-a631-fe81c9c15e5c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj" Jul 06 19:28:23.783614 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.783581 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj" Jul 06 19:28:23.921473 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.921434 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp"] Jul 06 19:28:23.923625 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.923584 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:23.927369 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.927335 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 19:28:23.927499 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.927374 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 19:28:23.927499 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.927376 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 19:28:23.927625 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.927611 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cvbvc\"" Jul 06 19:28:23.934488 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.934456 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 19:28:23.950936 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.950902 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp"] Jul 06 19:28:23.954536 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:23.954505 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj"] Jul 06 19:28:23.957839 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:23.957814 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47db4515_9fde_435f_a631_fe81c9c15e5c.slice/crio-3d4852e37a75d843e64af91d97885652b5378608a85c7b958400fc45e0267aba WatchSource:0}: Error finding container 3d4852e37a75d843e64af91d97885652b5378608a85c7b958400fc45e0267aba: Status 404 returned error can't find the container with id 3d4852e37a75d843e64af91d97885652b5378608a85c7b958400fc45e0267aba Jul 06 19:28:24.031008 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.030978 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-image-registry-private-configuration\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.031178 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.031018 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-certificates\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.031178 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.031045 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-bound-sa-token\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.031178 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.031107 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-ca-trust-extracted\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.031178 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.031140 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-trusted-ca\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.031344 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.031248 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.031344 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.031281 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5hqm\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-kube-api-access-b5hqm\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.031344 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.031315 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-installation-pull-secrets\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.131786 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.131742 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-certificates\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.131786 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.131791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-bound-sa-token\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132022 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.131819 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-ca-trust-extracted\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132022 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.131844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-trusted-ca\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132022 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.131926 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132022 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.131971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5hqm\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-kube-api-access-b5hqm\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132022 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.132000 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-installation-pull-secrets\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132286 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.132054 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-image-registry-private-configuration\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132286 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.132272 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-ca-trust-extracted\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132654 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:24.132281 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:28:24.132654 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:24.132450 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp: secret "image-registry-tls" not found Jul 06 19:28:24.132654 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.132429 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-certificates\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.132654 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:24.132504 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls podName:f6f3ea26-bdbe-465f-b94e-ce13cfb1821e nodeName:}" failed. No retries permitted until 2026-07-06 19:28:24.632485112 +0000 UTC m=+136.006941035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls") pod "image-registry-7d4fdbbfc7-wbvlp" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e") : secret "image-registry-tls" not found Jul 06 19:28:24.133289 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.133266 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-trusted-ca\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.135054 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.135009 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-image-registry-private-configuration\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.135183 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.135052 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-installation-pull-secrets\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.144625 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.144598 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5hqm\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-kube-api-access-b5hqm\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.146143 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.146107 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-bound-sa-token\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.637430 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.637390 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:24.637628 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:24.637535 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:28:24.637628 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:24.637557 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp: secret "image-registry-tls" not found Jul 06 19:28:24.637628 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:24.637608 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls podName:f6f3ea26-bdbe-465f-b94e-ce13cfb1821e nodeName:}" failed. No retries permitted until 2026-07-06 19:28:25.637593442 +0000 UTC m=+137.012049360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls") pod "image-registry-7d4fdbbfc7-wbvlp" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e") : secret "image-registry-tls" not found Jul 06 19:28:24.639942 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.639897 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj" event={"ID":"47db4515-9fde-435f-a631-fe81c9c15e5c","Type":"ContainerStarted","Data":"799541af733cba50b8ed75a6a17cac19b13384796c09a237b4e512cc682e2682"} Jul 06 19:28:24.640069 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.639955 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj" event={"ID":"47db4515-9fde-435f-a631-fe81c9c15e5c","Type":"ContainerStarted","Data":"3d4852e37a75d843e64af91d97885652b5378608a85c7b958400fc45e0267aba"} Jul 06 19:28:24.665897 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:24.665828 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d5shj" podStartSLOduration=1.6658064700000002 podStartE2EDuration="1.66580647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:24.664916946 +0000 UTC m=+136.039372886" watchObservedRunningTime="2026-07-06 19:28:24.66580647 +0000 UTC m=+136.040262411" Jul 06 19:28:25.243667 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.243622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:25.243876 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:25.243777 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:28:25.243876 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.243788 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:25.243876 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:25.243856 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:33.243835433 +0000 UTC m=+144.618291375 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : configmap "kube-rbac-proxy" not found Jul 06 19:28:25.244066 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:25.243944 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:28:25.244066 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:25.244007 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:33.243991367 +0000 UTC m=+144.618447285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : secret "insights-runtime-extractor-tls" not found Jul 06 19:28:25.644059 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.644023 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" event={"ID":"322e1481-512e-4f40-abcd-f12866faa4f4","Type":"ContainerStarted","Data":"5eba4ec879384ac515208d1eafd98f4a8399da93def2c198a43bb23a3d57c67c"} Jul 06 19:28:25.647689 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.647664 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:25.647796 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:25.647763 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:28:25.647796 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:25.647779 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp: secret "image-registry-tls" not found Jul 06 19:28:25.647858 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:25.647828 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls podName:f6f3ea26-bdbe-465f-b94e-ce13cfb1821e nodeName:}" failed. No retries permitted until 2026-07-06 19:28:27.647814932 +0000 UTC m=+139.022270849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls") pod "image-registry-7d4fdbbfc7-wbvlp" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e") : secret "image-registry-tls" not found Jul 06 19:28:25.668262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.668194 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" podStartSLOduration=1.483888637 podStartE2EDuration="3.668179982s" totalImagesPullingTime="2.184291345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:25.667431353 +0000 UTC m=+137.041887305" watchObservedRunningTime="2026-07-06 19:28:25.668179982 +0000 UTC m=+137.042635923" Jul 06 19:28:25.942043 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.942006 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c"] Jul 06 19:28:25.944298 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.944270 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" Jul 06 19:28:25.947139 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.947115 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 19:28:25.947280 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.947192 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 19:28:25.947339 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.947308 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pjgkq\"" Jul 06 19:28:25.955098 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:25.955076 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c"] Jul 06 19:28:26.050485 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:26.050453 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd9jd\" (UniqueName: \"kubernetes.io/projected/6053aed1-d818-4abb-8a87-8f9f5e0b2c8f-kube-api-access-kd9jd\") pod \"migrator-d57d58cc9-6544c\" (UID: \"6053aed1-d818-4abb-8a87-8f9f5e0b2c8f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" Jul 06 19:28:26.150981 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:26.150938 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd9jd\" (UniqueName: \"kubernetes.io/projected/6053aed1-d818-4abb-8a87-8f9f5e0b2c8f-kube-api-access-kd9jd\") pod \"migrator-d57d58cc9-6544c\" (UID: \"6053aed1-d818-4abb-8a87-8f9f5e0b2c8f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" Jul 06 19:28:26.161076 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:26.161048 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd9jd\" (UniqueName: \"kubernetes.io/projected/6053aed1-d818-4abb-8a87-8f9f5e0b2c8f-kube-api-access-kd9jd\") pod \"migrator-d57d58cc9-6544c\" (UID: \"6053aed1-d818-4abb-8a87-8f9f5e0b2c8f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" Jul 06 19:28:26.253338 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:26.253257 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" Jul 06 19:28:26.389353 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:26.389316 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c"] Jul 06 19:28:26.392022 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:26.391991 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6053aed1_d818_4abb_8a87_8f9f5e0b2c8f.slice/crio-e62c7a130abca285cb85772372c2cde8f5401324a276accf44ec9897ca9e0882 WatchSource:0}: Error finding container e62c7a130abca285cb85772372c2cde8f5401324a276accf44ec9897ca9e0882: Status 404 returned error can't find the container with id e62c7a130abca285cb85772372c2cde8f5401324a276accf44ec9897ca9e0882 Jul 06 19:28:26.648556 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:26.648523 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" event={"ID":"6053aed1-d818-4abb-8a87-8f9f5e0b2c8f","Type":"ContainerStarted","Data":"e62c7a130abca285cb85772372c2cde8f5401324a276accf44ec9897ca9e0882"} Jul 06 19:28:27.018632 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:27.018533 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:27.018632 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:27.018598 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:27.019094 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:27.018974 2565 scope.go:117] "RemoveContainer" containerID="c23f91e747b00c644cca3d041d7d31c3ed77905a5667aacb9c4d37f7fa5a7e57" Jul 06 19:28:27.019142 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:27.019128 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-6zvvd_openshift-console-operator(6f168e98-d917-4b91-9dfd-1a47192a428d)\"" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" podUID="6f168e98-d917-4b91-9dfd-1a47192a428d" Jul 06 19:28:27.663847 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:27.663813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:27.664018 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:27.663994 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:28:27.664082 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:27.664023 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp: secret "image-registry-tls" not found Jul 06 19:28:27.664119 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:27.664088 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls podName:f6f3ea26-bdbe-465f-b94e-ce13cfb1821e nodeName:}" failed. No retries permitted until 2026-07-06 19:28:31.664072998 +0000 UTC m=+143.038528943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls") pod "image-registry-7d4fdbbfc7-wbvlp" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e") : secret "image-registry-tls" not found Jul 06 19:28:28.656893 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:28.656858 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" event={"ID":"6053aed1-d818-4abb-8a87-8f9f5e0b2c8f","Type":"ContainerStarted","Data":"68d04cbc63a734e75b1e2c9e9e69193d6a19ebbeaf81d1ed83c0eed0325170f6"} Jul 06 19:28:28.657299 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:28.656939 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" event={"ID":"6053aed1-d818-4abb-8a87-8f9f5e0b2c8f","Type":"ContainerStarted","Data":"baea8c6b7908b43f71de21fe6975dc43cb2d6504894bc5e1a30dde5fcf10fbf8"} Jul 06 19:28:28.672635 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:28.672605 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:28.672731 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:28.672703 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:28.672774 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:28.672752 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls podName:cf9471e6-bb11-4bdf-b34e-f2821b1551a7 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:44.672738441 +0000 UTC m=+156.047194358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nct7w" (UID: "cf9471e6-bb11-4bdf-b34e-f2821b1551a7") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:28:28.675948 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:28.675906 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6544c" podStartSLOduration=2.241307625 podStartE2EDuration="3.675893128s" totalImagesPullingTime="1.434585503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:28.675040164 +0000 UTC m=+140.049496105" watchObservedRunningTime="2026-07-06 19:28:28.675893128 +0000 UTC m=+140.050349069" Jul 06 19:28:31.695102 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:31.695067 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:31.695511 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:31.695186 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:28:31.695511 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:31.695198 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp: secret "image-registry-tls" not found Jul 06 19:28:31.695511 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:31.695262 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls podName:f6f3ea26-bdbe-465f-b94e-ce13cfb1821e nodeName:}" failed. No retries permitted until 2026-07-06 19:28:39.695247947 +0000 UTC m=+151.069703864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls") pod "image-registry-7d4fdbbfc7-wbvlp" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e") : secret "image-registry-tls" not found Jul 06 19:28:33.301971 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:33.301928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:33.302399 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:33.301981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:33.302399 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:33.302086 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:28:33.302399 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:33.302163 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls podName:c3d34406-18c8-4adf-ba4b-4ed16cb0e367 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:49.302143097 +0000 UTC m=+160.676599032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbh7b" (UID: "c3d34406-18c8-4adf-ba4b-4ed16cb0e367") : secret "insights-runtime-extractor-tls" not found Jul 06 19:28:33.302508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:33.302490 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:39.747588 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:39.747529 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:39.749883 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:39.749860 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"image-registry-7d4fdbbfc7-wbvlp\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:39.845685 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:39.845637 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:39.995613 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:39.995575 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp"] Jul 06 19:28:39.999617 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:39.999550 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f3ea26_bdbe_465f_b94e_ce13cfb1821e.slice/crio-0e0c2a58691f36b49fc2e7229c794eb4dd58439af3ab2fc67df9fe7cb2ee860c WatchSource:0}: Error finding container 0e0c2a58691f36b49fc2e7229c794eb4dd58439af3ab2fc67df9fe7cb2ee860c: Status 404 returned error can't find the container with id 0e0c2a58691f36b49fc2e7229c794eb4dd58439af3ab2fc67df9fe7cb2ee860c Jul 06 19:28:40.698797 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:40.698761 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" event={"ID":"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e","Type":"ContainerStarted","Data":"01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70"} Jul 06 19:28:40.698990 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:40.698812 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" event={"ID":"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e","Type":"ContainerStarted","Data":"0e0c2a58691f36b49fc2e7229c794eb4dd58439af3ab2fc67df9fe7cb2ee860c"} Jul 06 19:28:40.698990 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:40.698913 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:28:40.730986 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:40.730936 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" podStartSLOduration=17.730920437 podStartE2EDuration="17.730920437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:40.730377529 +0000 UTC m=+152.104833498" watchObservedRunningTime="2026-07-06 19:28:40.730920437 +0000 UTC m=+152.105376394" Jul 06 19:28:41.199171 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:41.199135 2565 scope.go:117] "RemoveContainer" containerID="c23f91e747b00c644cca3d041d7d31c3ed77905a5667aacb9c4d37f7fa5a7e57" Jul 06 19:28:41.704349 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:41.704320 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:28:41.704509 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:41.704442 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" event={"ID":"6f168e98-d917-4b91-9dfd-1a47192a428d","Type":"ContainerStarted","Data":"8e41f3d87086846a690f726844e08c25e426ef5e2a2b6b4a831d05cdbe036631"} Jul 06 19:28:41.704784 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:41.704718 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:41.710969 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:41.710947 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" Jul 06 19:28:41.735356 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:41.735301 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-6zvvd" podStartSLOduration=23.62473228 podStartE2EDuration="25.735282931s" totalImagesPullingTime="2.110550651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:41.733576582 +0000 UTC m=+153.108032542" watchObservedRunningTime="2026-07-06 19:28:41.735282931 +0000 UTC m=+153.109738871" Jul 06 19:28:44.689036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:44.688988 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:44.691444 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:44.691411 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf9471e6-bb11-4bdf-b34e-f2821b1551a7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nct7w\" (UID: \"cf9471e6-bb11-4bdf-b34e-f2821b1551a7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:44.921093 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:44.921052 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" Jul 06 19:28:45.065501 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:45.065394 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w"] Jul 06 19:28:45.067991 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:45.067957 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf9471e6_bb11_4bdf_b34e_f2821b1551a7.slice/crio-4fd0cac2f15c6c967b4afe0e0d0740cb0fafe4b38b6e05a40c7f653e70d381f3 WatchSource:0}: Error finding container 4fd0cac2f15c6c967b4afe0e0d0740cb0fafe4b38b6e05a40c7f653e70d381f3: Status 404 returned error can't find the container with id 4fd0cac2f15c6c967b4afe0e0d0740cb0fafe4b38b6e05a40c7f653e70d381f3 Jul 06 19:28:45.718517 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:45.718480 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" event={"ID":"cf9471e6-bb11-4bdf-b34e-f2821b1551a7","Type":"ContainerStarted","Data":"4fd0cac2f15c6c967b4afe0e0d0740cb0fafe4b38b6e05a40c7f653e70d381f3"} Jul 06 19:28:46.496381 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:46.496334 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gggzl" podUID="9efbb99a-2d00-47b8-9188-4a35489cff63" Jul 06 19:28:46.724555 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:46.724492 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gggzl" Jul 06 19:28:47.728764 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:47.728724 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" event={"ID":"cf9471e6-bb11-4bdf-b34e-f2821b1551a7","Type":"ContainerStarted","Data":"e48d73a8bd3750cbbe1d54703e3b5c741f3267a29c46e6f8550e267d25928f32"} Jul 06 19:28:47.750701 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:47.750650 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nct7w" podStartSLOduration=34.114640148 podStartE2EDuration="35.750631668s" totalImagesPullingTime="1.63599152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:47.749763287 +0000 UTC m=+159.124219218" watchObservedRunningTime="2026-07-06 19:28:47.750631668 +0000 UTC m=+159.125087635" Jul 06 19:28:48.227338 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:28:48.227300 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5lcpf" podUID="14de08b3-3e7b-4641-97c8-7140cb4c24ec" Jul 06 19:28:49.326251 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:49.326200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:49.328436 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:49.328405 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3d34406-18c8-4adf-ba4b-4ed16cb0e367-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbh7b\" (UID: \"c3d34406-18c8-4adf-ba4b-4ed16cb0e367\") " pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:49.487465 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:49.487425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hbh7b" Jul 06 19:28:49.617197 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:49.617120 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hbh7b"] Jul 06 19:28:49.620615 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:49.620593 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d34406_18c8_4adf_ba4b_4ed16cb0e367.slice/crio-694781bbdd36a68bb22d30976c97ffb51b3f43533b320e284b00c25f69b310b3 WatchSource:0}: Error finding container 694781bbdd36a68bb22d30976c97ffb51b3f43533b320e284b00c25f69b310b3: Status 404 returned error can't find the container with id 694781bbdd36a68bb22d30976c97ffb51b3f43533b320e284b00c25f69b310b3 Jul 06 19:28:49.736230 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:49.736171 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hbh7b" event={"ID":"c3d34406-18c8-4adf-ba4b-4ed16cb0e367","Type":"ContainerStarted","Data":"694781bbdd36a68bb22d30976c97ffb51b3f43533b320e284b00c25f69b310b3"} Jul 06 19:28:51.443286 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:51.443238 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:28:51.445857 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:51.445830 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efbb99a-2d00-47b8-9188-4a35489cff63-metrics-tls\") pod \"dns-default-gggzl\" (UID: \"9efbb99a-2d00-47b8-9188-4a35489cff63\") " pod="openshift-dns/dns-default-gggzl" Jul 06 19:28:51.529103 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:51.529069 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7f4qs\"" Jul 06 19:28:51.536819 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:51.536786 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gggzl" Jul 06 19:28:52.852995 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.852961 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659"] Jul 06 19:28:52.865751 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.865717 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" Jul 06 19:28:52.871180 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.871148 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 19:28:52.871349 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.871148 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-szwkv\"" Jul 06 19:28:52.891673 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.891640 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp"] Jul 06 19:28:52.899838 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.899806 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659"] Jul 06 19:28:52.940564 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.940528 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8447b76dd-grsn2"] Jul 06 19:28:52.944112 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.944090 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:52.957639 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.957610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d24b55c0-3da1-4638-a4a2-f4a0df59fbcd-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-g6659\" (UID: \"d24b55c0-3da1-4638-a4a2-f4a0df59fbcd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" Jul 06 19:28:52.965599 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:52.965565 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8447b76dd-grsn2"] Jul 06 19:28:53.058413 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058364 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-installation-pull-secrets\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.058602 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058441 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnhnf\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-kube-api-access-lnhnf\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.058602 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058474 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-image-registry-private-configuration\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.058602 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058500 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-registry-tls\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.058602 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058581 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-ca-trust-extracted\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.058807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058609 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-registry-certificates\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.058807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d24b55c0-3da1-4638-a4a2-f4a0df59fbcd-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-g6659\" (UID: \"d24b55c0-3da1-4638-a4a2-f4a0df59fbcd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" Jul 06 19:28:53.058807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058732 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-trusted-ca\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.058807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.058764 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-bound-sa-token\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.061502 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.061472 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d24b55c0-3da1-4638-a4a2-f4a0df59fbcd-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-g6659\" (UID: \"d24b55c0-3da1-4638-a4a2-f4a0df59fbcd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" Jul 06 19:28:53.159882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.159800 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-installation-pull-secrets\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.159882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.159847 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnhnf\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-kube-api-access-lnhnf\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.159882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.159866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-image-registry-private-configuration\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.159882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.159882 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-registry-tls\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.160262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.159905 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-ca-trust-extracted\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.160262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.159924 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-registry-certificates\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.160262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.159963 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-trusted-ca\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.160262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.159995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-bound-sa-token\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.160717 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.160667 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-ca-trust-extracted\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.161005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.160975 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-registry-certificates\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.161316 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.161285 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-trusted-ca\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.162958 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.162935 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-installation-pull-secrets\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.162958 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.162949 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-registry-tls\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.163103 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.163038 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-image-registry-private-configuration\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.176509 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.176481 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" Jul 06 19:28:53.189086 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.189060 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-bound-sa-token\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.189614 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.189591 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnhnf\" (UniqueName: \"kubernetes.io/projected/e8c5b066-7aa9-487a-8dd1-54ce40af48c8-kube-api-access-lnhnf\") pod \"image-registry-8447b76dd-grsn2\" (UID: \"e8c5b066-7aa9-487a-8dd1-54ce40af48c8\") " pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.253065 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.253027 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.456056 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.456017 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8447b76dd-grsn2"] Jul 06 19:28:53.459891 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:53.459866 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c5b066_7aa9_487a_8dd1_54ce40af48c8.slice/crio-af41ec49b8114af2d4a8c6e76e2de24c1307914629a12e669e791f5538cc9de9 WatchSource:0}: Error finding container af41ec49b8114af2d4a8c6e76e2de24c1307914629a12e669e791f5538cc9de9: Status 404 returned error can't find the container with id af41ec49b8114af2d4a8c6e76e2de24c1307914629a12e669e791f5538cc9de9 Jul 06 19:28:53.469029 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.469002 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659"] Jul 06 19:28:53.472952 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:53.472925 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd24b55c0_3da1_4638_a4a2_f4a0df59fbcd.slice/crio-b718ea1f8fcb4a327a8de7911bf92bfae2f7ff0e0d5c99246af741619bb1b801 WatchSource:0}: Error finding container b718ea1f8fcb4a327a8de7911bf92bfae2f7ff0e0d5c99246af741619bb1b801: Status 404 returned error can't find the container with id b718ea1f8fcb4a327a8de7911bf92bfae2f7ff0e0d5c99246af741619bb1b801 Jul 06 19:28:53.492332 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.492304 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gggzl"] Jul 06 19:28:53.495656 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:53.495616 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efbb99a_2d00_47b8_9188_4a35489cff63.slice/crio-0b223aac629b051fc075d8b22fbee43dc02c992dedaccd5304a322d34a05f445 WatchSource:0}: Error finding container 0b223aac629b051fc075d8b22fbee43dc02c992dedaccd5304a322d34a05f445: Status 404 returned error can't find the container with id 0b223aac629b051fc075d8b22fbee43dc02c992dedaccd5304a322d34a05f445 Jul 06 19:28:53.752288 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.752178 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" event={"ID":"d24b55c0-3da1-4638-a4a2-f4a0df59fbcd","Type":"ContainerStarted","Data":"b718ea1f8fcb4a327a8de7911bf92bfae2f7ff0e0d5c99246af741619bb1b801"} Jul 06 19:28:53.753715 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.753678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hbh7b" event={"ID":"c3d34406-18c8-4adf-ba4b-4ed16cb0e367","Type":"ContainerStarted","Data":"48cd0d263af412c7490a99ad969aa63390f67edb6d0995d0471a52a9ec4265ca"} Jul 06 19:28:53.754802 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.754776 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gggzl" event={"ID":"9efbb99a-2d00-47b8-9188-4a35489cff63","Type":"ContainerStarted","Data":"0b223aac629b051fc075d8b22fbee43dc02c992dedaccd5304a322d34a05f445"} Jul 06 19:28:53.756257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.756230 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8447b76dd-grsn2" event={"ID":"e8c5b066-7aa9-487a-8dd1-54ce40af48c8","Type":"ContainerStarted","Data":"aba50d50c9778544904a11f205560c432fc9ff27bc689967d3b0b54647dce87c"} Jul 06 19:28:53.756359 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.756278 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8447b76dd-grsn2" event={"ID":"e8c5b066-7aa9-487a-8dd1-54ce40af48c8","Type":"ContainerStarted","Data":"af41ec49b8114af2d4a8c6e76e2de24c1307914629a12e669e791f5538cc9de9"} Jul 06 19:28:53.756428 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.756394 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:28:53.783816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:53.783748 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8447b76dd-grsn2" podStartSLOduration=1.783724904 podStartE2EDuration="1.783724904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:53.782636386 +0000 UTC m=+165.157092349" watchObservedRunningTime="2026-07-06 19:28:53.783724904 +0000 UTC m=+165.158180846" Jul 06 19:28:54.763298 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:54.763257 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hbh7b" event={"ID":"c3d34406-18c8-4adf-ba4b-4ed16cb0e367","Type":"ContainerStarted","Data":"1f626bca803bdff518f44bbd33abaab77bf74815e91c4c0e77127e9af87c5217"} Jul 06 19:28:55.768590 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:55.768547 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" event={"ID":"d24b55c0-3da1-4638-a4a2-f4a0df59fbcd","Type":"ContainerStarted","Data":"71b4cb1936882f8199a594e6a3a4fe0e7362d6ddb45515ca654874c2d4c63360"} Jul 06 19:28:55.769085 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:55.768737 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" Jul 06 19:28:55.770795 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:55.770764 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gggzl" event={"ID":"9efbb99a-2d00-47b8-9188-4a35489cff63","Type":"ContainerStarted","Data":"31318a57626c3ca37d4ff8bdd211ae988bc2de60d6f53bec422ff614d955db36"} Jul 06 19:28:55.770795 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:55.770818 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gggzl" event={"ID":"9efbb99a-2d00-47b8-9188-4a35489cff63","Type":"ContainerStarted","Data":"8104fcd98ae395b8ba5f5d4cbaa39460161f96a8285f94ab5447995d8ef1ea28"} Jul 06 19:28:55.770795 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:55.770848 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gggzl" Jul 06 19:28:55.774514 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:55.774480 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" Jul 06 19:28:55.792970 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:55.792920 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-g6659" podStartSLOduration=2.1324091960000002 podStartE2EDuration="3.792907071s" totalImagesPullingTime="1.660497875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:55.792389448 +0000 UTC m=+167.166845389" watchObservedRunningTime="2026-07-06 19:28:55.792907071 +0000 UTC m=+167.167363010" Jul 06 19:28:55.845812 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:55.845755 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gggzl" podStartSLOduration=131.202157585 podStartE2EDuration="2m12.845738749s" totalImagesPullingTime="1.643581164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:55.844190615 +0000 UTC m=+167.218646556" watchObservedRunningTime="2026-07-06 19:28:55.845738749 +0000 UTC m=+167.220194689" Jul 06 19:28:57.780137 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:57.780099 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hbh7b" event={"ID":"c3d34406-18c8-4adf-ba4b-4ed16cb0e367","Type":"ContainerStarted","Data":"7ee4240f1169d23b3c6bd9628a6ccab44c0f9c68d2c553a16fed0ed54c05e7ba"} Jul 06 19:28:57.807623 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:57.807572 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hbh7b" podStartSLOduration=34.384878589 podStartE2EDuration="41.807556067s" totalImagesPullingTime="7.422677478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:28:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:57.805344424 +0000 UTC m=+169.179800363" watchObservedRunningTime="2026-07-06 19:28:57.807556067 +0000 UTC m=+169.182012007" Jul 06 19:28:59.409297 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.409255 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-869b4b4cbf-pmhjs"] Jul 06 19:28:59.413952 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.413931 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.418028 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.417995 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 19:28:59.418315 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.418083 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 19:28:59.418711 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.418696 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rk2kv\"" Jul 06 19:28:59.419389 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.419370 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 19:28:59.419389 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.419385 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 19:28:59.419587 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.419405 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 19:28:59.419587 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.419495 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 19:28:59.428152 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.428133 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 19:28:59.428691 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.428672 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 19:28:59.429633 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.429614 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-869b4b4cbf-pmhjs"] Jul 06 19:28:59.511034 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.511004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-serving-cert\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.511234 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.511044 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-service-ca\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.511234 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.511080 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-oauth-serving-cert\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.511234 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.511129 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9hmt\" (UniqueName: \"kubernetes.io/projected/78628c93-d2ed-4718-920d-dae5f1ad932f-kube-api-access-d9hmt\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.511234 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.511155 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-oauth-config\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.511234 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.511178 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-trusted-ca-bundle\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.511485 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.511249 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-console-config\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.611920 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.611867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-serving-cert\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.611920 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.611918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-service-ca\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.612199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.611944 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-oauth-serving-cert\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.612199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.611964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9hmt\" (UniqueName: \"kubernetes.io/projected/78628c93-d2ed-4718-920d-dae5f1ad932f-kube-api-access-d9hmt\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.612199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.611981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-oauth-config\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.612199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.611995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-trusted-ca-bundle\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.612199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.612032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-console-config\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.612729 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.612706 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-service-ca\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.612925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.612900 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-oauth-serving-cert\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.612986 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.612949 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-console-config\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.613118 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.613100 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-trusted-ca-bundle\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.614468 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.614444 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-oauth-config\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.614563 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.614540 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-serving-cert\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.625451 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.625432 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9hmt\" (UniqueName: \"kubernetes.io/projected/78628c93-d2ed-4718-920d-dae5f1ad932f-kube-api-access-d9hmt\") pod \"console-869b4b4cbf-pmhjs\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.723245 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.723137 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:28:59.880562 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:28:59.880534 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-869b4b4cbf-pmhjs"] Jul 06 19:28:59.884060 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:28:59.884035 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78628c93_d2ed_4718_920d_dae5f1ad932f.slice/crio-8cb7acd7f0204a0d1d095fdc8de8887cc224a8c37d83480814c81e09980ac474 WatchSource:0}: Error finding container 8cb7acd7f0204a0d1d095fdc8de8887cc224a8c37d83480814c81e09980ac474: Status 404 returned error can't find the container with id 8cb7acd7f0204a0d1d095fdc8de8887cc224a8c37d83480814c81e09980ac474 Jul 06 19:29:00.795180 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:00.795127 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869b4b4cbf-pmhjs" event={"ID":"78628c93-d2ed-4718-920d-dae5f1ad932f","Type":"ContainerStarted","Data":"8cb7acd7f0204a0d1d095fdc8de8887cc224a8c37d83480814c81e09980ac474"} Jul 06 19:29:01.214945 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.214911 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rr2rs"] Jul 06 19:29:01.217924 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.217887 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.221187 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.221162 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:29:01.221439 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.221410 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7b468\"" Jul 06 19:29:01.221904 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.221457 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:29:01.221904 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.221468 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:29:01.221904 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.221782 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:29:01.325659 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325622 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-sys\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.325659 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325655 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-textfile\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.325866 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-wtmp\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.325866 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325725 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.325866 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-root\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.325866 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-metrics-client-ca\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.325866 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-tls\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.326060 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325877 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.326060 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.325903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czqm\" (UniqueName: \"kubernetes.io/projected/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-kube-api-access-5czqm\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427343 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427307 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427521 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427354 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-root\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427521 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427433 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-root\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427521 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427486 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-metrics-client-ca\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-tls\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427594 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427623 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czqm\" (UniqueName: \"kubernetes.io/projected/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-kube-api-access-5czqm\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427693 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:29:01.427656 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 19:29:01.427693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427673 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-sys\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427921 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-textfile\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.427921 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:29:01.427731 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-tls podName:7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 19:29:01.927711618 +0000 UTC m=+173.302167547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-tls") pod "node-exporter-rr2rs" (UID: "7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0") : secret "node-exporter-tls" not found Jul 06 19:29:01.427921 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427765 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-wtmp\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.428062 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427926 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-wtmp\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.428062 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.427955 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.428151 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.428062 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-sys\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.428441 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.428021 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-textfile\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.428548 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.428518 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-metrics-client-ca\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.433800 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.433742 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.449042 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.449012 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czqm\" (UniqueName: \"kubernetes.io/projected/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-kube-api-access-5czqm\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.932862 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.932817 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-tls\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:01.935645 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:01.935609 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0-node-exporter-tls\") pod \"node-exporter-rr2rs\" (UID: \"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0\") " pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:02.129520 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.129481 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rr2rs" Jul 06 19:29:02.231600 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.231513 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:29:02.237028 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.236996 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.242452 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242242 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:29:02.242452 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242272 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-thr8m\"" Jul 06 19:29:02.242452 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242312 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:29:02.242452 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242320 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:29:02.242452 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242352 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:29:02.242452 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242272 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:29:02.242452 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242397 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:29:02.242823 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242518 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:29:02.242823 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242567 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:29:02.242823 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.242584 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:29:02.263473 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.263441 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:29:02.337229 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337402 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337237 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337402 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337371 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-web-config\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337470 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-out\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337541 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337594 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337615 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t89sn\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-kube-api-access-t89sn\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.337901 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.337728 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.392225 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:29:02.392172 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f85b68f_47c6_47f6_be2d_6f04f6e8f0d0.slice/crio-4fad60bfa7d29c5af8f391d8b94e75802614458d6894a56d9c15f17d41a26651 WatchSource:0}: Error finding container 4fad60bfa7d29c5af8f391d8b94e75802614458d6894a56d9c15f17d41a26651: Status 404 returned error can't find the container with id 4fad60bfa7d29c5af8f391d8b94e75802614458d6894a56d9c15f17d41a26651 Jul 06 19:29:02.438634 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.438610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.438756 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.438652 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.438756 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.438682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-web-config\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.438756 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.438705 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.438926 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.438880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-out\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.438926 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.438917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.439008 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.438950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.439008 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.438989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.439099 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.439012 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t89sn\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-kube-api-access-t89sn\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.439099 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.439041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.439199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.439095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.439199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.439124 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.439199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.439155 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.439490 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.439464 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.441229 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.440662 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.441229 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.440914 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.441771 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.441749 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-out\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.442040 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.442017 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.442123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.442081 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.442162 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.442132 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.442346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.442321 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.442989 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.442934 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.443098 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.443076 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-web-config\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.444054 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.444026 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.444152 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.444034 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.452745 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.452720 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t89sn\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-kube-api-access-t89sn\") pod \"alertmanager-main-0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.548281 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.548238 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:29:02.701489 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.701458 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:29:02.703911 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:29:02.703874 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9816f6d1_0a16_42f3_a93e_9373aa3826b0.slice/crio-c375e66aa275217514af4894913019101dccd1332e6b17e211d99c113fdb4c05 WatchSource:0}: Error finding container c375e66aa275217514af4894913019101dccd1332e6b17e211d99c113fdb4c05: Status 404 returned error can't find the container with id c375e66aa275217514af4894913019101dccd1332e6b17e211d99c113fdb4c05 Jul 06 19:29:02.805850 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.805757 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerStarted","Data":"c375e66aa275217514af4894913019101dccd1332e6b17e211d99c113fdb4c05"} Jul 06 19:29:02.807265 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.807230 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rr2rs" event={"ID":"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0","Type":"ContainerStarted","Data":"4fad60bfa7d29c5af8f391d8b94e75802614458d6894a56d9c15f17d41a26651"} Jul 06 19:29:02.809285 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.809254 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869b4b4cbf-pmhjs" event={"ID":"78628c93-d2ed-4718-920d-dae5f1ad932f","Type":"ContainerStarted","Data":"1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10"} Jul 06 19:29:02.834381 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.834314 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-869b4b4cbf-pmhjs" podStartSLOduration=1.283687194 podStartE2EDuration="3.83429756s" totalImagesPullingTime="2.550610366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:29:02.832328988 +0000 UTC m=+174.206784927" watchObservedRunningTime="2026-07-06 19:29:02.83429756 +0000 UTC m=+174.208753499" Jul 06 19:29:02.897916 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:02.897882 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:29:03.204420 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:03.204394 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:29:03.815619 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:03.815582 2565 generic.go:356] "Generic (PLEG): container finished" podID="7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0" containerID="142423f4342f868e643f2bf609cfbcd8c8aed49b2ada04ef4b5580517bd268b4" exitCode=0 Jul 06 19:29:03.815784 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:03.815686 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rr2rs" event={"ID":"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0","Type":"ContainerDied","Data":"142423f4342f868e643f2bf609cfbcd8c8aed49b2ada04ef4b5580517bd268b4"} Jul 06 19:29:04.820676 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:04.820636 2565 generic.go:356] "Generic (PLEG): container finished" podID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerID="f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60" exitCode=0 Jul 06 19:29:04.821112 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:04.820726 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerDied","Data":"f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60"} Jul 06 19:29:04.823106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:04.823073 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rr2rs" event={"ID":"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0","Type":"ContainerStarted","Data":"7e30f54f485304e163c031036d83335ebb53a0ff881f0147b5014256a2fd45d7"} Jul 06 19:29:04.823206 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:04.823123 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rr2rs" event={"ID":"7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0","Type":"ContainerStarted","Data":"470fbdbefbae65664c81cd7596864e27772230daf6e25abf1dbf0c3cb4869a70"} Jul 06 19:29:04.881674 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:04.881617 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rr2rs" podStartSLOduration=2.9413620849999997 podStartE2EDuration="3.881599813s" totalImagesPullingTime="872.040408ms" totalInitContainerRuntime="68.19732ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:29:04.880424907 +0000 UTC m=+176.254880847" watchObservedRunningTime="2026-07-06 19:29:04.881599813 +0000 UTC m=+176.256055752" Jul 06 19:29:05.768795 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.768763 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv"] Jul 06 19:29:05.772175 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.772151 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:05.774984 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.774962 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 19:29:05.775102 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.774985 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vp442\"" Jul 06 19:29:05.777174 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.777151 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gggzl" Jul 06 19:29:05.786417 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.786393 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv"] Jul 06 19:29:05.873183 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.873133 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1ed35988-0b83-46eb-ae76-40c4e5857726-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-kmjwv\" (UID: \"1ed35988-0b83-46eb-ae76-40c4e5857726\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:05.873630 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.873208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmds7\" (UniqueName: \"kubernetes.io/projected/1ed35988-0b83-46eb-ae76-40c4e5857726-kube-api-access-fmds7\") pod \"monitoring-plugin-d5dcff7cf-kmjwv\" (UID: \"1ed35988-0b83-46eb-ae76-40c4e5857726\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:05.973883 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.973849 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1ed35988-0b83-46eb-ae76-40c4e5857726-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-kmjwv\" (UID: \"1ed35988-0b83-46eb-ae76-40c4e5857726\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:05.974055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.973909 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmds7\" (UniqueName: \"kubernetes.io/projected/1ed35988-0b83-46eb-ae76-40c4e5857726-kube-api-access-fmds7\") pod \"monitoring-plugin-d5dcff7cf-kmjwv\" (UID: \"1ed35988-0b83-46eb-ae76-40c4e5857726\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:05.976567 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.976528 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1ed35988-0b83-46eb-ae76-40c4e5857726-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-kmjwv\" (UID: \"1ed35988-0b83-46eb-ae76-40c4e5857726\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:05.987109 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:05.987080 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmds7\" (UniqueName: \"kubernetes.io/projected/1ed35988-0b83-46eb-ae76-40c4e5857726-kube-api-access-fmds7\") pod \"monitoring-plugin-d5dcff7cf-kmjwv\" (UID: \"1ed35988-0b83-46eb-ae76-40c4e5857726\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:06.083031 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.082997 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:06.226774 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.226741 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-869b4b4cbf-pmhjs"] Jul 06 19:29:06.420534 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.420511 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv"] Jul 06 19:29:06.423702 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:29:06.423673 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ed35988_0b83_46eb_ae76_40c4e5857726.slice/crio-c966917ff7e4a1093ffddc7b7c4767dfdf3cae14699f4f10c255edeecf54ba07 WatchSource:0}: Error finding container c966917ff7e4a1093ffddc7b7c4767dfdf3cae14699f4f10c255edeecf54ba07: Status 404 returned error can't find the container with id c966917ff7e4a1093ffddc7b7c4767dfdf3cae14699f4f10c255edeecf54ba07 Jul 06 19:29:06.832693 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.832653 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" event={"ID":"1ed35988-0b83-46eb-ae76-40c4e5857726","Type":"ContainerStarted","Data":"c966917ff7e4a1093ffddc7b7c4767dfdf3cae14699f4f10c255edeecf54ba07"} Jul 06 19:29:06.835704 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.835676 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerStarted","Data":"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da"} Jul 06 19:29:06.835845 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.835737 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerStarted","Data":"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f"} Jul 06 19:29:06.835845 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.835762 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerStarted","Data":"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54"} Jul 06 19:29:06.835845 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.835786 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerStarted","Data":"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7"} Jul 06 19:29:06.835845 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:06.835812 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerStarted","Data":"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411"} Jul 06 19:29:07.571061 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.571026 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:29:07.576327 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.576305 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.579263 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579236 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:29:07.579442 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579407 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:29:07.579674 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579456 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pzcpz\"" Jul 06 19:29:07.579674 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579640 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 19:29:07.579825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579688 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:29:07.579825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579643 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:29:07.579825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579642 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:29:07.579825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579812 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:29:07.580019 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.579643 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:29:07.580019 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.580005 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:29:07.580173 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.580155 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:29:07.580848 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.580825 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:29:07.580949 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.580824 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:29:07.580949 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.580862 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:29:07.580949 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.580827 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5s952bd4tlqmp\"" Jul 06 19:29:07.584506 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.584475 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:29:07.588050 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.588028 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:29:07.594874 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.594852 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:29:07.699145 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699112 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699145 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699368 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699181 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699368 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699203 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config-out\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699368 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-web-config\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699368 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699315 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699484 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699368 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699484 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699396 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699484 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699433 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699484 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699457 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699484 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699477 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9w6w\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-kube-api-access-f9w6w\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699643 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699502 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699643 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699569 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699643 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699596 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699643 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699616 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699643 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699638 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699663 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.699807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.699717 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800553 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800519 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9w6w\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-kube-api-access-f9w6w\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800592 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800642 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800710 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800699 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800736 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800764 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800857 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.800966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800940 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.800985 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.801031 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config-out\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.801065 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-web-config\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.801099 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.801144 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.801175 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.801239 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801676 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.801503 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.801825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.801803 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.803689 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.803662 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.803794 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.803688 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.803860 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.803831 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.803969 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.803909 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.803969 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.803950 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.804419 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.804397 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.804555 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.804466 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.804677 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.804654 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config-out\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.804742 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.804661 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.805233 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.805156 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.805476 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.805451 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.805604 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.805585 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.806505 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.806464 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.807054 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.807031 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.807233 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.807198 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.807289 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.807261 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-web-config\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.807627 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.807604 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.822111 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.822046 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9w6w\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-kube-api-access-f9w6w\") pod \"prometheus-k8s-0\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:07.886981 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:07.886951 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:08.039080 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.039047 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:29:08.042285 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:29:08.042255 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb428a9c7_6b1b_42ff_974d_7dd72b99bb22.slice/crio-c1af1575569524e0efa313303cc6c257cac9dbcfcfcd51951edb6d3f13d504e3 WatchSource:0}: Error finding container c1af1575569524e0efa313303cc6c257cac9dbcfcfcd51951edb6d3f13d504e3: Status 404 returned error can't find the container with id c1af1575569524e0efa313303cc6c257cac9dbcfcfcd51951edb6d3f13d504e3 Jul 06 19:29:08.844238 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.844181 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" event={"ID":"1ed35988-0b83-46eb-ae76-40c4e5857726","Type":"ContainerStarted","Data":"118f045ae55de7d6b9467d8e1f7f74f831e9805a56667d975daf98fb24439f84"} Jul 06 19:29:08.844692 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.844267 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:08.845818 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.845786 2565 generic.go:356] "Generic (PLEG): container finished" podID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" exitCode=0 Jul 06 19:29:08.845950 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.845864 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerDied","Data":"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb"} Jul 06 19:29:08.845950 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.845920 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerStarted","Data":"c1af1575569524e0efa313303cc6c257cac9dbcfcfcd51951edb6d3f13d504e3"} Jul 06 19:29:08.849648 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.849625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerStarted","Data":"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6"} Jul 06 19:29:08.850392 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.850366 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" Jul 06 19:29:08.865098 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.865059 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-kmjwv" podStartSLOduration=2.5315770730000002 podStartE2EDuration="3.865040811s" totalImagesPullingTime="1.333463738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:29:08.864638058 +0000 UTC m=+180.239093997" watchObservedRunningTime="2026-07-06 19:29:08.865040811 +0000 UTC m=+180.239496752" Jul 06 19:29:08.909750 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:08.909686 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.021537581 podStartE2EDuration="6.909667176s" totalImagesPullingTime="3.871514185s" totalInitContainerRuntime="16.61541ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:29:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:29:08.902998312 +0000 UTC m=+180.277454254" watchObservedRunningTime="2026-07-06 19:29:08.909667176 +0000 UTC m=+180.284123119" Jul 06 19:29:09.724262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:09.724195 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:29:12.868012 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:12.867976 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerStarted","Data":"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c"} Jul 06 19:29:12.868408 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:12.868045 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerStarted","Data":"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c"} Jul 06 19:29:14.768476 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:14.768449 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8447b76dd-grsn2" Jul 06 19:29:14.882909 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:14.882869 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerStarted","Data":"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7"} Jul 06 19:29:14.883102 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:14.882969 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerStarted","Data":"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b"} Jul 06 19:29:14.883102 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:14.882998 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerStarted","Data":"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c"} Jul 06 19:29:14.883102 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:14.883022 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerStarted","Data":"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6"} Jul 06 19:29:14.923396 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:14.923299 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.493810326 podStartE2EDuration="7.923281436s" totalImagesPullingTime="5.40743366s" totalInitContainerRuntime="22.03745ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:29:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:29:14.921184843 +0000 UTC m=+186.295640850" watchObservedRunningTime="2026-07-06 19:29:14.923281436 +0000 UTC m=+186.297737377" Jul 06 19:29:17.888078 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:17.888043 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:17.914590 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:17.914549 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" podUID="f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" containerName="registry" containerID="cri-o://01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70" gracePeriod=30 Jul 06 19:29:18.162097 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.162071 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:29:18.215781 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.215745 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-image-registry-private-configuration\") pod \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " Jul 06 19:29:18.215781 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.215785 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-trusted-ca\") pod \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " Jul 06 19:29:18.215993 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.215867 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-installation-pull-secrets\") pod \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " Jul 06 19:29:18.216257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.216041 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-ca-trust-extracted\") pod \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " Jul 06 19:29:18.216257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.216084 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-certificates\") pod \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " Jul 06 19:29:18.216257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.216120 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-bound-sa-token\") pod \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " Jul 06 19:29:18.216257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.216160 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") pod \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " Jul 06 19:29:18.216257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.216191 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5hqm\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-kube-api-access-b5hqm\") pod \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\" (UID: \"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e\") " Jul 06 19:29:18.216257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.216237 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:18.216578 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.216528 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-trusted-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:18.216578 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.216522 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:18.218772 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.218741 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:18.218867 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.218790 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:18.218867 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.218812 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:18.218951 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.218900 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-kube-api-access-b5hqm" (OuterVolumeSpecName: "kube-api-access-b5hqm") pod "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e"). InnerVolumeSpecName "kube-api-access-b5hqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:18.219106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.219081 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:18.225129 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.225103 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" (UID: "f6f3ea26-bdbe-465f-b94e-ce13cfb1821e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:18.317607 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.317573 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-installation-pull-secrets\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:18.317607 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.317605 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-ca-trust-extracted\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:18.317607 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.317616 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-certificates\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:18.317839 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.317625 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-bound-sa-token\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:18.317839 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.317634 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-registry-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:18.317839 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.317643 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5hqm\" (UniqueName: \"kubernetes.io/projected/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-kube-api-access-b5hqm\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:18.317839 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.317652 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e-image-registry-private-configuration\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:18.899405 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.899367 2565 generic.go:356] "Generic (PLEG): container finished" podID="f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" containerID="01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70" exitCode=0 Jul 06 19:29:18.899794 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.899430 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" Jul 06 19:29:18.899794 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.899441 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" event={"ID":"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e","Type":"ContainerDied","Data":"01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70"} Jul 06 19:29:18.899794 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.899505 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp" event={"ID":"f6f3ea26-bdbe-465f-b94e-ce13cfb1821e","Type":"ContainerDied","Data":"0e0c2a58691f36b49fc2e7229c794eb4dd58439af3ab2fc67df9fe7cb2ee860c"} Jul 06 19:29:18.899794 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.899524 2565 scope.go:117] "RemoveContainer" containerID="01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70" Jul 06 19:29:18.908415 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.908359 2565 scope.go:117] "RemoveContainer" containerID="01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70" Jul 06 19:29:18.908679 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:29:18.908657 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70\": container with ID starting with 01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70 not found: ID does not exist" containerID="01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70" Jul 06 19:29:18.908754 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.908713 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70"} err="failed to get container status \"01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70\": rpc error: code = NotFound desc = could not find container \"01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70\": container with ID starting with 01feb205b2276f17d0c3926b2af9d9d2a76cbf0feb974d98631754995063ee70 not found: ID does not exist" Jul 06 19:29:18.926095 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.926061 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp"] Jul 06 19:29:18.931962 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:18.931936 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d4fdbbfc7-wbvlp"] Jul 06 19:29:19.204124 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:19.204045 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" path="/var/lib/kubelet/pods/f6f3ea26-bdbe-465f-b94e-ce13cfb1821e/volumes" Jul 06 19:29:31.252200 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.252124 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-869b4b4cbf-pmhjs" podUID="78628c93-d2ed-4718-920d-dae5f1ad932f" containerName="console" containerID="cri-o://1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10" gracePeriod=15 Jul 06 19:29:31.504550 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.504477 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:29:31.640587 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.640551 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-serving-cert\") pod \"78628c93-d2ed-4718-920d-dae5f1ad932f\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " Jul 06 19:29:31.640587 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.640593 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-oauth-serving-cert\") pod \"78628c93-d2ed-4718-920d-dae5f1ad932f\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " Jul 06 19:29:31.640819 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.640617 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-trusted-ca-bundle\") pod \"78628c93-d2ed-4718-920d-dae5f1ad932f\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " Jul 06 19:29:31.640819 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.640650 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-oauth-config\") pod \"78628c93-d2ed-4718-920d-dae5f1ad932f\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " Jul 06 19:29:31.640819 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.640682 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-console-config\") pod \"78628c93-d2ed-4718-920d-dae5f1ad932f\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " Jul 06 19:29:31.640819 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.640714 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9hmt\" (UniqueName: \"kubernetes.io/projected/78628c93-d2ed-4718-920d-dae5f1ad932f-kube-api-access-d9hmt\") pod \"78628c93-d2ed-4718-920d-dae5f1ad932f\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " Jul 06 19:29:31.640819 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.640739 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-service-ca\") pod \"78628c93-d2ed-4718-920d-dae5f1ad932f\" (UID: \"78628c93-d2ed-4718-920d-dae5f1ad932f\") " Jul 06 19:29:31.641101 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.641049 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "78628c93-d2ed-4718-920d-dae5f1ad932f" (UID: "78628c93-d2ed-4718-920d-dae5f1ad932f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:31.641101 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.641072 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "78628c93-d2ed-4718-920d-dae5f1ad932f" (UID: "78628c93-d2ed-4718-920d-dae5f1ad932f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:31.641243 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.641141 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-console-config" (OuterVolumeSpecName: "console-config") pod "78628c93-d2ed-4718-920d-dae5f1ad932f" (UID: "78628c93-d2ed-4718-920d-dae5f1ad932f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:31.641340 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.641318 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-service-ca" (OuterVolumeSpecName: "service-ca") pod "78628c93-d2ed-4718-920d-dae5f1ad932f" (UID: "78628c93-d2ed-4718-920d-dae5f1ad932f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:31.643096 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.643073 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "78628c93-d2ed-4718-920d-dae5f1ad932f" (UID: "78628c93-d2ed-4718-920d-dae5f1ad932f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:31.643096 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.643080 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78628c93-d2ed-4718-920d-dae5f1ad932f-kube-api-access-d9hmt" (OuterVolumeSpecName: "kube-api-access-d9hmt") pod "78628c93-d2ed-4718-920d-dae5f1ad932f" (UID: "78628c93-d2ed-4718-920d-dae5f1ad932f"). InnerVolumeSpecName "kube-api-access-d9hmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:31.643245 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.643206 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "78628c93-d2ed-4718-920d-dae5f1ad932f" (UID: "78628c93-d2ed-4718-920d-dae5f1ad932f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:31.742320 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.742283 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-serving-cert\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:31.742320 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.742311 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-oauth-serving-cert\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:31.742320 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.742321 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-trusted-ca-bundle\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:31.742320 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.742329 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78628c93-d2ed-4718-920d-dae5f1ad932f-console-oauth-config\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:31.742589 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.742338 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-console-config\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:31.742589 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.742346 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9hmt\" (UniqueName: \"kubernetes.io/projected/78628c93-d2ed-4718-920d-dae5f1ad932f-kube-api-access-d9hmt\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:31.742589 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.742355 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78628c93-d2ed-4718-920d-dae5f1ad932f-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:29:31.951728 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.951688 2565 generic.go:356] "Generic (PLEG): container finished" podID="78628c93-d2ed-4718-920d-dae5f1ad932f" containerID="1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10" exitCode=0 Jul 06 19:29:31.951921 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.951742 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869b4b4cbf-pmhjs" Jul 06 19:29:31.951921 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.951765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869b4b4cbf-pmhjs" event={"ID":"78628c93-d2ed-4718-920d-dae5f1ad932f","Type":"ContainerDied","Data":"1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10"} Jul 06 19:29:31.951921 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.951817 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869b4b4cbf-pmhjs" event={"ID":"78628c93-d2ed-4718-920d-dae5f1ad932f","Type":"ContainerDied","Data":"8cb7acd7f0204a0d1d095fdc8de8887cc224a8c37d83480814c81e09980ac474"} Jul 06 19:29:31.951921 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.951844 2565 scope.go:117] "RemoveContainer" containerID="1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10" Jul 06 19:29:31.960716 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.960696 2565 scope.go:117] "RemoveContainer" containerID="1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10" Jul 06 19:29:31.961023 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:29:31.961002 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10\": container with ID starting with 1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10 not found: ID does not exist" containerID="1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10" Jul 06 19:29:31.961078 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.961033 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10"} err="failed to get container status \"1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10\": rpc error: code = NotFound desc = could not find container \"1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10\": container with ID starting with 1020698f38e9a191919f6c54b0eb2fec2b04e66f8a56c8a842da1caab445eb10 not found: ID does not exist" Jul 06 19:29:31.978661 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.978617 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-869b4b4cbf-pmhjs"] Jul 06 19:29:31.989534 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:31.989502 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-869b4b4cbf-pmhjs"] Jul 06 19:29:33.203033 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:33.202994 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78628c93-d2ed-4718-920d-dae5f1ad932f" path="/var/lib/kubelet/pods/78628c93-d2ed-4718-920d-dae5f1ad932f/volumes" Jul 06 19:29:33.961486 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:33.961448 2565 generic.go:356] "Generic (PLEG): container finished" podID="543e3ca8-22d4-4b02-989f-af18444f9f57" containerID="585b35edc544b83144d905a92672b24a9e1e35ae0dffed9e43c3dae3a9fa2566" exitCode=0 Jul 06 19:29:33.961657 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:33.961515 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" event={"ID":"543e3ca8-22d4-4b02-989f-af18444f9f57","Type":"ContainerDied","Data":"585b35edc544b83144d905a92672b24a9e1e35ae0dffed9e43c3dae3a9fa2566"} Jul 06 19:29:33.961900 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:33.961885 2565 scope.go:117] "RemoveContainer" containerID="585b35edc544b83144d905a92672b24a9e1e35ae0dffed9e43c3dae3a9fa2566" Jul 06 19:29:34.967049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:34.967008 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hrn4g" event={"ID":"543e3ca8-22d4-4b02-989f-af18444f9f57","Type":"ContainerStarted","Data":"13aad5a14c1e788ae2718fd0669eca834d1b2abb9ad11cd100bfd960e3c310fc"} Jul 06 19:29:47.017454 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:47.017416 2565 generic.go:356] "Generic (PLEG): container finished" podID="322e1481-512e-4f40-abcd-f12866faa4f4" containerID="5eba4ec879384ac515208d1eafd98f4a8399da93def2c198a43bb23a3d57c67c" exitCode=0 Jul 06 19:29:47.017985 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:47.017495 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" event={"ID":"322e1481-512e-4f40-abcd-f12866faa4f4","Type":"ContainerDied","Data":"5eba4ec879384ac515208d1eafd98f4a8399da93def2c198a43bb23a3d57c67c"} Jul 06 19:29:47.017985 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:47.017927 2565 scope.go:117] "RemoveContainer" containerID="5eba4ec879384ac515208d1eafd98f4a8399da93def2c198a43bb23a3d57c67c" Jul 06 19:29:48.022784 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:29:48.022747 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-skfd7" event={"ID":"322e1481-512e-4f40-abcd-f12866faa4f4","Type":"ContainerStarted","Data":"dbfc37ec2b6ce34c29cf474736d52809a3b21a7fb2fab7947d1763ae597c1c1b"} Jul 06 19:30:07.887387 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:07.887348 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:07.908833 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:07.908805 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:08.125500 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:08.125474 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:21.080518 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.080480 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:30:21.082844 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.082817 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14de08b3-3e7b-4641-97c8-7140cb4c24ec-metrics-certs\") pod \"network-metrics-daemon-5lcpf\" (UID: \"14de08b3-3e7b-4641-97c8-7140cb4c24ec\") " pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:30:21.208358 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.208330 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t5qdm\"" Jul 06 19:30:21.216271 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.216251 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5lcpf" Jul 06 19:30:21.366353 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.366298 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5lcpf"] Jul 06 19:30:21.368997 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:30:21.368972 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14de08b3_3e7b_4641_97c8_7140cb4c24ec.slice/crio-0200639fd31b15f6e7636db10d3846d609ad291bd300115cbc54746a61d95834 WatchSource:0}: Error finding container 0200639fd31b15f6e7636db10d3846d609ad291bd300115cbc54746a61d95834: Status 404 returned error can't find the container with id 0200639fd31b15f6e7636db10d3846d609ad291bd300115cbc54746a61d95834 Jul 06 19:30:21.792535 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.792443 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:21.792812 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.792786 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="alertmanager" containerID="cri-o://16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411" gracePeriod=120 Jul 06 19:30:21.792911 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.792853 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy-metric" containerID="cri-o://f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da" gracePeriod=120 Jul 06 19:30:21.792911 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.792897 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="config-reloader" containerID="cri-o://79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7" gracePeriod=120 Jul 06 19:30:21.793014 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.792891 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy" containerID="cri-o://8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f" gracePeriod=120 Jul 06 19:30:21.793014 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.792857 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy-web" containerID="cri-o://50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54" gracePeriod=120 Jul 06 19:30:21.793014 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:21.792958 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="prom-label-proxy" containerID="cri-o://18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6" gracePeriod=120 Jul 06 19:30:22.167733 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.167696 2565 generic.go:356] "Generic (PLEG): container finished" podID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerID="18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6" exitCode=0 Jul 06 19:30:22.167733 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.167728 2565 generic.go:356] "Generic (PLEG): container finished" podID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerID="8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f" exitCode=0 Jul 06 19:30:22.167733 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.167737 2565 generic.go:356] "Generic (PLEG): container finished" podID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerID="79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7" exitCode=0 Jul 06 19:30:22.167733 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.167743 2565 generic.go:356] "Generic (PLEG): container finished" podID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerID="16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411" exitCode=0 Jul 06 19:30:22.168351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.167779 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerDied","Data":"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6"} Jul 06 19:30:22.168351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.167867 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerDied","Data":"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f"} Jul 06 19:30:22.168351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.167910 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerDied","Data":"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7"} Jul 06 19:30:22.168351 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.167953 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerDied","Data":"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411"} Jul 06 19:30:22.169240 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:22.169191 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5lcpf" event={"ID":"14de08b3-3e7b-4641-97c8-7140cb4c24ec","Type":"ContainerStarted","Data":"0200639fd31b15f6e7636db10d3846d609ad291bd300115cbc54746a61d95834"} Jul 06 19:30:23.024406 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.024372 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:23.102891 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.102801 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-volume\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.102891 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.102854 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103101 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103030 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-web-config\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103101 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103090 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103197 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103138 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-tls-assets\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103197 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103176 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-main-db\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103340 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103235 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-main-tls\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103340 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103267 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t89sn\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-kube-api-access-t89sn\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103340 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103312 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103490 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103367 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-trusted-ca-bundle\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103490 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103393 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-metrics-client-ca\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103490 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103419 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-out\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.103490 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.103465 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-cluster-tls-config\") pod \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\" (UID: \"9816f6d1-0a16-42f3-a93e-9373aa3826b0\") " Jul 06 19:30:23.104685 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.104631 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:23.104817 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.104685 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:23.104817 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.104742 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:23.105692 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.105665 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:23.107373 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.107346 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-volume" (OuterVolumeSpecName: "config-volume") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:23.108006 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.107958 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-kube-api-access-t89sn" (OuterVolumeSpecName: "kube-api-access-t89sn") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "kube-api-access-t89sn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:23.108006 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.107995 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:23.108388 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.108355 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:23.108849 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.108819 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:23.109499 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.109478 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-out" (OuterVolumeSpecName: "config-out") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:23.110229 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.110190 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:23.113278 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.113254 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:23.120056 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.120029 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-web-config" (OuterVolumeSpecName: "web-config") pod "9816f6d1-0a16-42f3-a93e-9373aa3826b0" (UID: "9816f6d1-0a16-42f3-a93e-9373aa3826b0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:23.175367 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.175329 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5lcpf" event={"ID":"14de08b3-3e7b-4641-97c8-7140cb4c24ec","Type":"ContainerStarted","Data":"2deb38b601ce03373a121b94d56b950cc0914021963fc7e3ba43dab928f9bc86"} Jul 06 19:30:23.175806 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.175408 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5lcpf" event={"ID":"14de08b3-3e7b-4641-97c8-7140cb4c24ec","Type":"ContainerStarted","Data":"8bbf40189942bc69e69736617371c538aa239e58ae6e2017ad8589f4a0f28143"} Jul 06 19:30:23.178812 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.178784 2565 generic.go:356] "Generic (PLEG): container finished" podID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerID="f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da" exitCode=0 Jul 06 19:30:23.178812 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.178808 2565 generic.go:356] "Generic (PLEG): container finished" podID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerID="50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54" exitCode=0 Jul 06 19:30:23.178997 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.178854 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:23.178997 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.178852 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerDied","Data":"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da"} Jul 06 19:30:23.179094 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.179015 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerDied","Data":"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54"} Jul 06 19:30:23.179094 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.179064 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9816f6d1-0a16-42f3-a93e-9373aa3826b0","Type":"ContainerDied","Data":"c375e66aa275217514af4894913019101dccd1332e6b17e211d99c113fdb4c05"} Jul 06 19:30:23.179094 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.179076 2565 scope.go:117] "RemoveContainer" containerID="18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6" Jul 06 19:30:23.187314 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.187278 2565 scope.go:117] "RemoveContainer" containerID="f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da" Jul 06 19:30:23.194368 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.194352 2565 scope.go:117] "RemoveContainer" containerID="8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f" Jul 06 19:30:23.202634 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.202617 2565 scope.go:117] "RemoveContainer" containerID="50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54" Jul 06 19:30:23.204815 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204795 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t89sn\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-kube-api-access-t89sn\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204817 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204827 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204837 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9816f6d1-0a16-42f3-a93e-9373aa3826b0-metrics-client-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204847 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-out\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204855 2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-cluster-tls-config\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204864 2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-config-volume\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204873 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204882 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-web-config\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204890 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.204896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204897 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9816f6d1-0a16-42f3-a93e-9373aa3826b0-tls-assets\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.205261 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204906 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9816f6d1-0a16-42f3-a93e-9373aa3826b0-alertmanager-main-db\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.205261 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.204914 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9816f6d1-0a16-42f3-a93e-9373aa3826b0-secret-alertmanager-main-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.210025 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.210006 2565 scope.go:117] "RemoveContainer" containerID="79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7" Jul 06 19:30:23.212080 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.212031 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5lcpf" podStartSLOduration=253.105297981 podStartE2EDuration="4m14.212018758s" totalImagesPullingTime="1.106720777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:23.21141317 +0000 UTC m=+254.585869109" watchObservedRunningTime="2026-07-06 19:30:23.212018758 +0000 UTC m=+254.586474697" Jul 06 19:30:23.217263 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.217242 2565 scope.go:117] "RemoveContainer" containerID="16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411" Jul 06 19:30:23.224748 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.224732 2565 scope.go:117] "RemoveContainer" containerID="f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60" Jul 06 19:30:23.231806 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.231787 2565 scope.go:117] "RemoveContainer" containerID="18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6" Jul 06 19:30:23.232064 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:23.232044 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6\": container with ID starting with 18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6 not found: ID does not exist" containerID="18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6" Jul 06 19:30:23.232110 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.232072 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6"} err="failed to get container status \"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6\": rpc error: code = NotFound desc = could not find container \"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6\": container with ID starting with 18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6 not found: ID does not exist" Jul 06 19:30:23.232110 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.232089 2565 scope.go:117] "RemoveContainer" containerID="f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da" Jul 06 19:30:23.232338 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:23.232323 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da\": container with ID starting with f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da not found: ID does not exist" containerID="f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da" Jul 06 19:30:23.232384 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.232342 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da"} err="failed to get container status \"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da\": rpc error: code = NotFound desc = could not find container \"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da\": container with ID starting with f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da not found: ID does not exist" Jul 06 19:30:23.232384 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.232373 2565 scope.go:117] "RemoveContainer" containerID="8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f" Jul 06 19:30:23.232597 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:23.232581 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f\": container with ID starting with 8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f not found: ID does not exist" containerID="8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f" Jul 06 19:30:23.232642 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.232600 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f"} err="failed to get container status \"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f\": rpc error: code = NotFound desc = could not find container \"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f\": container with ID starting with 8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f not found: ID does not exist" Jul 06 19:30:23.232642 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.232612 2565 scope.go:117] "RemoveContainer" containerID="50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54" Jul 06 19:30:23.232826 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:23.232804 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54\": container with ID starting with 50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54 not found: ID does not exist" containerID="50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54" Jul 06 19:30:23.232871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.232829 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54"} err="failed to get container status \"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54\": rpc error: code = NotFound desc = could not find container \"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54\": container with ID starting with 50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54 not found: ID does not exist" Jul 06 19:30:23.232871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.232841 2565 scope.go:117] "RemoveContainer" containerID="79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7" Jul 06 19:30:23.233016 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:23.233002 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7\": container with ID starting with 79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7 not found: ID does not exist" containerID="79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7" Jul 06 19:30:23.233049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233019 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7"} err="failed to get container status \"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7\": rpc error: code = NotFound desc = could not find container \"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7\": container with ID starting with 79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7 not found: ID does not exist" Jul 06 19:30:23.233049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233030 2565 scope.go:117] "RemoveContainer" containerID="16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411" Jul 06 19:30:23.233219 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:23.233194 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411\": container with ID starting with 16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411 not found: ID does not exist" containerID="16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411" Jul 06 19:30:23.233260 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233232 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411"} err="failed to get container status \"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411\": rpc error: code = NotFound desc = could not find container \"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411\": container with ID starting with 16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411 not found: ID does not exist" Jul 06 19:30:23.233260 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233245 2565 scope.go:117] "RemoveContainer" containerID="f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60" Jul 06 19:30:23.233422 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:23.233407 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60\": container with ID starting with f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60 not found: ID does not exist" containerID="f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60" Jul 06 19:30:23.233464 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233425 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60"} err="failed to get container status \"f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60\": rpc error: code = NotFound desc = could not find container \"f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60\": container with ID starting with f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60 not found: ID does not exist" Jul 06 19:30:23.233464 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233438 2565 scope.go:117] "RemoveContainer" containerID="18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6" Jul 06 19:30:23.233605 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233590 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6"} err="failed to get container status \"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6\": rpc error: code = NotFound desc = could not find container \"18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6\": container with ID starting with 18dcdabae135ef10182ed2a8b6aa62d8d03b088436f69d7d95424e14478ca5f6 not found: ID does not exist" Jul 06 19:30:23.233605 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233605 2565 scope.go:117] "RemoveContainer" containerID="f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da" Jul 06 19:30:23.233763 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233749 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da"} err="failed to get container status \"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da\": rpc error: code = NotFound desc = could not find container \"f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da\": container with ID starting with f15d48d69119c66c7c836420aaa3e2f97ea4cfa90ee5821546e1a79c770db5da not found: ID does not exist" Jul 06 19:30:23.233763 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233762 2565 scope.go:117] "RemoveContainer" containerID="8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f" Jul 06 19:30:23.233944 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233923 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f"} err="failed to get container status \"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f\": rpc error: code = NotFound desc = could not find container \"8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f\": container with ID starting with 8c0c8593715a56af42cf0d031321940a59bfb98f20ad6f59fc8f2fc8b4d5e79f not found: ID does not exist" Jul 06 19:30:23.233984 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.233946 2565 scope.go:117] "RemoveContainer" containerID="50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54" Jul 06 19:30:23.234185 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.234161 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54"} err="failed to get container status \"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54\": rpc error: code = NotFound desc = could not find container \"50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54\": container with ID starting with 50b2c56df5c5e057983dc060b6a6f24cec255c91d3d44d1f0bee0bf26de8cd54 not found: ID does not exist" Jul 06 19:30:23.234185 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.234183 2565 scope.go:117] "RemoveContainer" containerID="79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7" Jul 06 19:30:23.234439 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.234412 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7"} err="failed to get container status \"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7\": rpc error: code = NotFound desc = could not find container \"79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7\": container with ID starting with 79cb6084ee1ab4228239e5d0a6697c1d7bed2a268229dabdb291709b5f597ec7 not found: ID does not exist" Jul 06 19:30:23.234486 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.234441 2565 scope.go:117] "RemoveContainer" containerID="16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411" Jul 06 19:30:23.234655 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.234639 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411"} err="failed to get container status \"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411\": rpc error: code = NotFound desc = could not find container \"16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411\": container with ID starting with 16f732537456b931b12bbacc35d743284bb10d0213314d1334fd72078a56d411 not found: ID does not exist" Jul 06 19:30:23.234705 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.234655 2565 scope.go:117] "RemoveContainer" containerID="f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60" Jul 06 19:30:23.234833 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.234813 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60"} err="failed to get container status \"f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60\": rpc error: code = NotFound desc = could not find container \"f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60\": container with ID starting with f9a3f878b5309981c36b10e492f9617d1479f16682a61ef7acf5647314af9e60 not found: ID does not exist" Jul 06 19:30:23.265896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.265849 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:23.267054 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:23.267030 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:25.202709 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:25.202674 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" path="/var/lib/kubelet/pods/9816f6d1-0a16-42f3-a93e-9373aa3826b0/volumes" Jul 06 19:30:26.092206 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.092165 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:26.092513 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.092484 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="prometheus" containerID="cri-o://f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" gracePeriod=600 Jul 06 19:30:26.092597 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.092522 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="config-reloader" containerID="cri-o://2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" gracePeriod=600 Jul 06 19:30:26.092597 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.092543 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" gracePeriod=600 Jul 06 19:30:26.092698 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.092497 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy" containerID="cri-o://fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" gracePeriod=600 Jul 06 19:30:26.092698 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.092554 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy-web" containerID="cri-o://11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" gracePeriod=600 Jul 06 19:30:26.092698 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.092525 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="thanos-sidecar" containerID="cri-o://de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" gracePeriod=600 Jul 06 19:30:26.357316 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.357292 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:26.437465 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437434 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-thanos-prometheus-http-client-file\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437626 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437496 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-web-config\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437626 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437543 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-grpc-tls\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437626 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437586 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config-out\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437759 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437619 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-db\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437759 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437667 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437759 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437698 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-serving-certs-ca-bundle\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437759 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437745 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-tls-assets\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437960 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437771 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-metrics-client-certs\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437960 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437796 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-metrics-client-ca\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437960 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437820 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-1\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437960 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437859 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-0\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437960 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437883 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9w6w\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-kube-api-access-f9w6w\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437960 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437918 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.437960 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437940 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-trusted-ca-bundle\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.438330 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437969 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-2\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.438330 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.437994 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-kubelet-serving-ca-bundle\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.438330 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.438032 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-tls\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.438330 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.438066 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-kube-rbac-proxy\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.438330 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.438103 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\" (UID: \"b428a9c7-6b1b-42ff-974d-7dd72b99bb22\") " Jul 06 19:30:26.438330 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.438197 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:26.438643 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.438396 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.438643 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.438432 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:26.439817 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.439789 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:26.440077 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.440047 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:26.440432 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.440345 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:26.440965 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.440930 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:26.441542 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.441356 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:26.441542 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.441488 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.441542 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.441512 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config-out" (OuterVolumeSpecName: "config-out") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:26.441920 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.441850 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.441920 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.441881 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config" (OuterVolumeSpecName: "config") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.441920 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.441889 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:26.441920 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.441916 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.442822 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.442786 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:26.443700 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.443670 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-kube-api-access-f9w6w" (OuterVolumeSpecName: "kube-api-access-f9w6w") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "kube-api-access-f9w6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:26.443787 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.443703 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.443787 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.443714 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.443787 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.443734 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.444188 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.444170 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.452505 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.452481 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-web-config" (OuterVolumeSpecName: "web-config") pod "b428a9c7-6b1b-42ff-974d-7dd72b99bb22" (UID: "b428a9c7-6b1b-42ff-974d-7dd72b99bb22"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:26.538997 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.538955 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.538997 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.538987 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.538997 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.538997 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.538997 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539008 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539020 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539029 2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-kube-rbac-proxy\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539037 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539046 2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539056 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-web-config\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539065 2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-grpc-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539072 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config-out\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539080 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-db\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539087 2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-config\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539096 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539106 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-tls-assets\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539114 2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-secret-metrics-client-certs\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539122 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-configmap-metrics-client-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539144 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:26.539292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:26.539154 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9w6w\" (UniqueName: \"kubernetes.io/projected/b428a9c7-6b1b-42ff-974d-7dd72b99bb22-kube-api-access-f9w6w\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:30:27.199874 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.199839 2565 generic.go:356] "Generic (PLEG): container finished" podID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" exitCode=0 Jul 06 19:30:27.199874 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.199868 2565 generic.go:356] "Generic (PLEG): container finished" podID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" exitCode=0 Jul 06 19:30:27.199874 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.199880 2565 generic.go:356] "Generic (PLEG): container finished" podID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" exitCode=0 Jul 06 19:30:27.200091 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.199889 2565 generic.go:356] "Generic (PLEG): container finished" podID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" exitCode=0 Jul 06 19:30:27.200091 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.199898 2565 generic.go:356] "Generic (PLEG): container finished" podID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" exitCode=0 Jul 06 19:30:27.200091 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.199906 2565 generic.go:356] "Generic (PLEG): container finished" podID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" exitCode=0 Jul 06 19:30:27.200091 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.199971 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.203936 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.203909 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerDied","Data":"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7"} Jul 06 19:30:27.204059 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.203966 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerDied","Data":"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b"} Jul 06 19:30:27.204059 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.204003 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerDied","Data":"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c"} Jul 06 19:30:27.204059 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.204034 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerDied","Data":"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6"} Jul 06 19:30:27.204059 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.204045 2565 scope.go:117] "RemoveContainer" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" Jul 06 19:30:27.204199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.204071 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerDied","Data":"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c"} Jul 06 19:30:27.204199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.204111 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerDied","Data":"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c"} Jul 06 19:30:27.204199 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.204151 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b428a9c7-6b1b-42ff-974d-7dd72b99bb22","Type":"ContainerDied","Data":"c1af1575569524e0efa313303cc6c257cac9dbcfcfcd51951edb6d3f13d504e3"} Jul 06 19:30:27.213252 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.213235 2565 scope.go:117] "RemoveContainer" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" Jul 06 19:30:27.220138 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.220121 2565 scope.go:117] "RemoveContainer" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" Jul 06 19:30:27.226639 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.226622 2565 scope.go:117] "RemoveContainer" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" Jul 06 19:30:27.233569 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.233547 2565 scope.go:117] "RemoveContainer" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" Jul 06 19:30:27.236800 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.236775 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:27.239152 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.239133 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:27.241988 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.241971 2565 scope.go:117] "RemoveContainer" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" Jul 06 19:30:27.249109 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.249092 2565 scope.go:117] "RemoveContainer" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" Jul 06 19:30:27.255692 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.255676 2565 scope.go:117] "RemoveContainer" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" Jul 06 19:30:27.255970 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:27.255951 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": container with ID starting with 3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7 not found: ID does not exist" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" Jul 06 19:30:27.256024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.255982 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7"} err="failed to get container status \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": rpc error: code = NotFound desc = could not find container \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": container with ID starting with 3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7 not found: ID does not exist" Jul 06 19:30:27.256024 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.256006 2565 scope.go:117] "RemoveContainer" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" Jul 06 19:30:27.256284 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:27.256266 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": container with ID starting with fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b not found: ID does not exist" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" Jul 06 19:30:27.256346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.256290 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b"} err="failed to get container status \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": rpc error: code = NotFound desc = could not find container \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": container with ID starting with fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b not found: ID does not exist" Jul 06 19:30:27.256346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.256307 2565 scope.go:117] "RemoveContainer" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" Jul 06 19:30:27.256560 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:27.256543 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": container with ID starting with 11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c not found: ID does not exist" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" Jul 06 19:30:27.256601 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.256565 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c"} err="failed to get container status \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": rpc error: code = NotFound desc = could not find container \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": container with ID starting with 11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c not found: ID does not exist" Jul 06 19:30:27.256601 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.256580 2565 scope.go:117] "RemoveContainer" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" Jul 06 19:30:27.256811 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:27.256795 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": container with ID starting with de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6 not found: ID does not exist" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" Jul 06 19:30:27.256871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.256819 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6"} err="failed to get container status \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": rpc error: code = NotFound desc = could not find container \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": container with ID starting with de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6 not found: ID does not exist" Jul 06 19:30:27.256871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.256840 2565 scope.go:117] "RemoveContainer" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" Jul 06 19:30:27.257072 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:27.257058 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": container with ID starting with 2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c not found: ID does not exist" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" Jul 06 19:30:27.257112 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.257075 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c"} err="failed to get container status \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": rpc error: code = NotFound desc = could not find container \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": container with ID starting with 2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c not found: ID does not exist" Jul 06 19:30:27.257112 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.257089 2565 scope.go:117] "RemoveContainer" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" Jul 06 19:30:27.257340 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:27.257322 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": container with ID starting with f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c not found: ID does not exist" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" Jul 06 19:30:27.257384 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.257346 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c"} err="failed to get container status \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": rpc error: code = NotFound desc = could not find container \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": container with ID starting with f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c not found: ID does not exist" Jul 06 19:30:27.257384 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.257360 2565 scope.go:117] "RemoveContainer" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" Jul 06 19:30:27.257612 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:30:27.257597 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": container with ID starting with 1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb not found: ID does not exist" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" Jul 06 19:30:27.257656 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.257615 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb"} err="failed to get container status \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": rpc error: code = NotFound desc = could not find container \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": container with ID starting with 1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb not found: ID does not exist" Jul 06 19:30:27.257656 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.257627 2565 scope.go:117] "RemoveContainer" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" Jul 06 19:30:27.257879 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.257861 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7"} err="failed to get container status \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": rpc error: code = NotFound desc = could not find container \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": container with ID starting with 3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7 not found: ID does not exist" Jul 06 19:30:27.257946 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.257881 2565 scope.go:117] "RemoveContainer" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" Jul 06 19:30:27.258114 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258096 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b"} err="failed to get container status \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": rpc error: code = NotFound desc = could not find container \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": container with ID starting with fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b not found: ID does not exist" Jul 06 19:30:27.258153 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258114 2565 scope.go:117] "RemoveContainer" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" Jul 06 19:30:27.258369 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258346 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c"} err="failed to get container status \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": rpc error: code = NotFound desc = could not find container \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": container with ID starting with 11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c not found: ID does not exist" Jul 06 19:30:27.258369 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258369 2565 scope.go:117] "RemoveContainer" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" Jul 06 19:30:27.258591 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258574 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6"} err="failed to get container status \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": rpc error: code = NotFound desc = could not find container \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": container with ID starting with de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6 not found: ID does not exist" Jul 06 19:30:27.258637 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258592 2565 scope.go:117] "RemoveContainer" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" Jul 06 19:30:27.258780 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258763 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c"} err="failed to get container status \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": rpc error: code = NotFound desc = could not find container \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": container with ID starting with 2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c not found: ID does not exist" Jul 06 19:30:27.258822 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258784 2565 scope.go:117] "RemoveContainer" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" Jul 06 19:30:27.259013 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.258998 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c"} err="failed to get container status \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": rpc error: code = NotFound desc = could not find container \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": container with ID starting with f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c not found: ID does not exist" Jul 06 19:30:27.259054 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259012 2565 scope.go:117] "RemoveContainer" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" Jul 06 19:30:27.259198 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259178 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb"} err="failed to get container status \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": rpc error: code = NotFound desc = could not find container \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": container with ID starting with 1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb not found: ID does not exist" Jul 06 19:30:27.259261 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259199 2565 scope.go:117] "RemoveContainer" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" Jul 06 19:30:27.259397 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259382 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7"} err="failed to get container status \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": rpc error: code = NotFound desc = could not find container \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": container with ID starting with 3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7 not found: ID does not exist" Jul 06 19:30:27.259437 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259397 2565 scope.go:117] "RemoveContainer" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" Jul 06 19:30:27.259615 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259597 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b"} err="failed to get container status \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": rpc error: code = NotFound desc = could not find container \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": container with ID starting with fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b not found: ID does not exist" Jul 06 19:30:27.259668 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259615 2565 scope.go:117] "RemoveContainer" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" Jul 06 19:30:27.259787 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259770 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c"} err="failed to get container status \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": rpc error: code = NotFound desc = could not find container \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": container with ID starting with 11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c not found: ID does not exist" Jul 06 19:30:27.259870 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259789 2565 scope.go:117] "RemoveContainer" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" Jul 06 19:30:27.259964 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259948 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6"} err="failed to get container status \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": rpc error: code = NotFound desc = could not find container \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": container with ID starting with de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6 not found: ID does not exist" Jul 06 19:30:27.260005 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.259966 2565 scope.go:117] "RemoveContainer" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" Jul 06 19:30:27.260141 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260125 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c"} err="failed to get container status \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": rpc error: code = NotFound desc = could not find container \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": container with ID starting with 2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c not found: ID does not exist" Jul 06 19:30:27.260180 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260141 2565 scope.go:117] "RemoveContainer" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" Jul 06 19:30:27.260360 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260342 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c"} err="failed to get container status \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": rpc error: code = NotFound desc = could not find container \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": container with ID starting with f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c not found: ID does not exist" Jul 06 19:30:27.260417 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260361 2565 scope.go:117] "RemoveContainer" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" Jul 06 19:30:27.260572 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260556 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb"} err="failed to get container status \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": rpc error: code = NotFound desc = could not find container \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": container with ID starting with 1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb not found: ID does not exist" Jul 06 19:30:27.260572 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260571 2565 scope.go:117] "RemoveContainer" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" Jul 06 19:30:27.260802 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260787 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7"} err="failed to get container status \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": rpc error: code = NotFound desc = could not find container \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": container with ID starting with 3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7 not found: ID does not exist" Jul 06 19:30:27.260839 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260803 2565 scope.go:117] "RemoveContainer" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" Jul 06 19:30:27.261008 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.260991 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b"} err="failed to get container status \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": rpc error: code = NotFound desc = could not find container \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": container with ID starting with fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b not found: ID does not exist" Jul 06 19:30:27.261054 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261008 2565 scope.go:117] "RemoveContainer" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" Jul 06 19:30:27.261222 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261193 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c"} err="failed to get container status \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": rpc error: code = NotFound desc = could not find container \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": container with ID starting with 11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c not found: ID does not exist" Jul 06 19:30:27.261268 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261224 2565 scope.go:117] "RemoveContainer" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" Jul 06 19:30:27.261421 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261404 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6"} err="failed to get container status \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": rpc error: code = NotFound desc = could not find container \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": container with ID starting with de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6 not found: ID does not exist" Jul 06 19:30:27.261526 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261431 2565 scope.go:117] "RemoveContainer" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" Jul 06 19:30:27.261660 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261641 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c"} err="failed to get container status \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": rpc error: code = NotFound desc = could not find container \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": container with ID starting with 2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c not found: ID does not exist" Jul 06 19:30:27.261714 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261660 2565 scope.go:117] "RemoveContainer" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" Jul 06 19:30:27.261903 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261881 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c"} err="failed to get container status \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": rpc error: code = NotFound desc = could not find container \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": container with ID starting with f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c not found: ID does not exist" Jul 06 19:30:27.262010 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.261988 2565 scope.go:117] "RemoveContainer" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" Jul 06 19:30:27.262266 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.262248 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb"} err="failed to get container status \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": rpc error: code = NotFound desc = could not find container \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": container with ID starting with 1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb not found: ID does not exist" Jul 06 19:30:27.262266 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.262266 2565 scope.go:117] "RemoveContainer" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" Jul 06 19:30:27.262493 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.262478 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7"} err="failed to get container status \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": rpc error: code = NotFound desc = could not find container \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": container with ID starting with 3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7 not found: ID does not exist" Jul 06 19:30:27.262596 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.262494 2565 scope.go:117] "RemoveContainer" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" Jul 06 19:30:27.262722 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.262704 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b"} err="failed to get container status \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": rpc error: code = NotFound desc = could not find container \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": container with ID starting with fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b not found: ID does not exist" Jul 06 19:30:27.262773 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.262722 2565 scope.go:117] "RemoveContainer" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" Jul 06 19:30:27.262936 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.262919 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c"} err="failed to get container status \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": rpc error: code = NotFound desc = could not find container \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": container with ID starting with 11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c not found: ID does not exist" Jul 06 19:30:27.262973 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.262936 2565 scope.go:117] "RemoveContainer" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" Jul 06 19:30:27.263138 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.263124 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6"} err="failed to get container status \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": rpc error: code = NotFound desc = could not find container \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": container with ID starting with de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6 not found: ID does not exist" Jul 06 19:30:27.263176 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.263138 2565 scope.go:117] "RemoveContainer" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" Jul 06 19:30:27.263366 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.263347 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c"} err="failed to get container status \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": rpc error: code = NotFound desc = could not find container \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": container with ID starting with 2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c not found: ID does not exist" Jul 06 19:30:27.263423 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.263366 2565 scope.go:117] "RemoveContainer" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" Jul 06 19:30:27.263583 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.263569 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c"} err="failed to get container status \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": rpc error: code = NotFound desc = could not find container \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": container with ID starting with f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c not found: ID does not exist" Jul 06 19:30:27.263619 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.263583 2565 scope.go:117] "RemoveContainer" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" Jul 06 19:30:27.268421 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.266498 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb"} err="failed to get container status \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": rpc error: code = NotFound desc = could not find container \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": container with ID starting with 1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb not found: ID does not exist" Jul 06 19:30:27.268421 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.266532 2565 scope.go:117] "RemoveContainer" containerID="3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7" Jul 06 19:30:27.268838 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.268808 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7"} err="failed to get container status \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": rpc error: code = NotFound desc = could not find container \"3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7\": container with ID starting with 3b0b8f364a484d3056b27c8e4fe4150e06f2e42d90e5ed0fd0dde6d65d6e61e7 not found: ID does not exist" Jul 06 19:30:27.268879 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.268841 2565 scope.go:117] "RemoveContainer" containerID="fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b" Jul 06 19:30:27.269128 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.269093 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b"} err="failed to get container status \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": rpc error: code = NotFound desc = could not find container \"fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b\": container with ID starting with fc73f3d1f6ed6d7bb198caca5a49406f9fb298ce49d658eaf6419b97bfb4032b not found: ID does not exist" Jul 06 19:30:27.269189 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.269129 2565 scope.go:117] "RemoveContainer" containerID="11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c" Jul 06 19:30:27.269419 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.269398 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c"} err="failed to get container status \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": rpc error: code = NotFound desc = could not find container \"11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c\": container with ID starting with 11ae2bbec6442039efe7cf37a709a20d8d4e53501716e3946d5ce754085ab32c not found: ID does not exist" Jul 06 19:30:27.269419 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.269418 2565 scope.go:117] "RemoveContainer" containerID="de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6" Jul 06 19:30:27.269639 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.269622 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6"} err="failed to get container status \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": rpc error: code = NotFound desc = could not find container \"de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6\": container with ID starting with de40761ff7653e84a953feeff93dfe7af35eeafa8d269cb9557943b54ac899a6 not found: ID does not exist" Jul 06 19:30:27.269684 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.269640 2565 scope.go:117] "RemoveContainer" containerID="2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c" Jul 06 19:30:27.269879 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.269859 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c"} err="failed to get container status \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": rpc error: code = NotFound desc = could not find container \"2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c\": container with ID starting with 2dc8ad86171609b88b19ed9e22040539687bd1085dd057dd1cd1cecbd96cdb9c not found: ID does not exist" Jul 06 19:30:27.269958 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.269881 2565 scope.go:117] "RemoveContainer" containerID="f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c" Jul 06 19:30:27.270134 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.270109 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c"} err="failed to get container status \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": rpc error: code = NotFound desc = could not find container \"f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c\": container with ID starting with f16889467c10e91b14c6ad49ab5da826d9ed8fa7a6c086ca5eff8919485b740c not found: ID does not exist" Jul 06 19:30:27.270182 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.270135 2565 scope.go:117] "RemoveContainer" containerID="1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb" Jul 06 19:30:27.270452 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.270404 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb"} err="failed to get container status \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": rpc error: code = NotFound desc = could not find container \"1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb\": container with ID starting with 1422d69d5dc1155e827c8ba230f00fbd8804e6c54178d67fb23b67494c8c4edb not found: ID does not exist" Jul 06 19:30:27.290149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290120 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:27.290428 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290408 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="init-config-reloader" Jul 06 19:30:27.290428 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290429 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="init-config-reloader" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290439 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="init-config-reloader" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290445 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="init-config-reloader" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290454 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy-web" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290459 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy-web" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290465 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy-thanos" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290471 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy-thanos" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290479 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78628c93-d2ed-4718-920d-dae5f1ad932f" containerName="console" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290484 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="78628c93-d2ed-4718-920d-dae5f1ad932f" containerName="console" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290489 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy-web" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290494 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy-web" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290502 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="prom-label-proxy" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290507 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="prom-label-proxy" Jul 06 19:30:27.290508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290513 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="thanos-sidecar" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290518 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="thanos-sidecar" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290526 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" containerName="registry" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290532 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" containerName="registry" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290542 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="config-reloader" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290546 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="config-reloader" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290551 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290556 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290562 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="prometheus" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290567 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="prometheus" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290575 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="config-reloader" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290580 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="config-reloader" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290585 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="alertmanager" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290590 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="alertmanager" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290596 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290601 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290605 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy-metric" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290610 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy-metric" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290663 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy-web" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290670 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="thanos-sidecar" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290677 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="config-reloader" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290682 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy-thanos" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290687 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="78628c93-d2ed-4718-920d-dae5f1ad932f" containerName="console" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290691 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="prometheus" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290696 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="prom-label-proxy" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290701 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy-web" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290706 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290710 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="config-reloader" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290715 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" containerName="kube-rbac-proxy" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290720 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="kube-rbac-proxy-metric" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290725 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6f3ea26-bdbe-465f-b94e-ce13cfb1821e" containerName="registry" Jul 06 19:30:27.290871 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.290729 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9816f6d1-0a16-42f3-a93e-9373aa3826b0" containerName="alertmanager" Jul 06 19:30:27.296123 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.296099 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.299730 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.299706 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:30:27.299836 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.299791 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:30:27.300396 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.300366 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:30:27.300502 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.300424 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:30:27.300564 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.300514 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:30:27.300711 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.300696 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:30:27.300866 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.300851 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pzcpz\"" Jul 06 19:30:27.300946 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.300928 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:30:27.301398 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.301381 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:30:27.301481 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.301401 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:30:27.301481 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.301382 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:30:27.301923 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.301724 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:30:27.301923 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.301760 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:30:27.301923 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.301787 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5s952bd4tlqmp\"" Jul 06 19:30:27.302135 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.302118 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 19:30:27.304651 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.304625 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:30:27.308119 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.308101 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:30:27.327069 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.327042 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:27.446346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446259 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8df2fa50-9067-4ed0-93bc-7743cacb7424-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446293 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446317 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446360 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446389 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446434 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjwnk\" (UniqueName: \"kubernetes.io/projected/8df2fa50-9067-4ed0-93bc-7743cacb7424-kube-api-access-wjwnk\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446502 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446526 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446572 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-web-config\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446608 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446632 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446647 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446678 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8df2fa50-9067-4ed0-93bc-7743cacb7424-config-out\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446716 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446735 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446765 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-config\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.446825 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446780 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.447336 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.446795 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547286 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547339 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-web-config\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547361 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547658 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547383 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547658 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547399 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547658 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8df2fa50-9067-4ed0-93bc-7743cacb7424-config-out\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547658 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547658 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547578 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547658 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547603 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547658 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-config\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.547658 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547663 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547685 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8df2fa50-9067-4ed0-93bc-7743cacb7424-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547732 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547768 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjwnk\" (UniqueName: \"kubernetes.io/projected/8df2fa50-9067-4ed0-93bc-7743cacb7424-kube-api-access-wjwnk\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547879 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.547905 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548049 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.548011 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.548586 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.548171 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.549588 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.548664 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.549588 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.549284 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.550655 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.550632 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.551383 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.551355 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8df2fa50-9067-4ed0-93bc-7743cacb7424-config-out\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.551615 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.551591 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8df2fa50-9067-4ed0-93bc-7743cacb7424-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.551685 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.551646 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-web-config\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.551749 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.551687 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.552316 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.551971 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.552316 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.552110 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.552316 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.552174 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-config\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.552316 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.552409 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.552316 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.552497 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.552854 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.552737 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.553641 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.553619 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.554444 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.554410 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.554652 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.554631 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8df2fa50-9067-4ed0-93bc-7743cacb7424-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.555290 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.555270 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8df2fa50-9067-4ed0-93bc-7743cacb7424-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.566118 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.566094 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjwnk\" (UniqueName: \"kubernetes.io/projected/8df2fa50-9067-4ed0-93bc-7743cacb7424-kube-api-access-wjwnk\") pod \"prometheus-k8s-0\" (UID: \"8df2fa50-9067-4ed0-93bc-7743cacb7424\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.606240 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.606179 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:27.756606 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:27.756527 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:27.759619 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:30:27.759593 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df2fa50_9067_4ed0_93bc_7743cacb7424.slice/crio-ffe89225ffc8480b499d233d345c24684fa7687c0aa66d9b03e4fa746ebb592c WatchSource:0}: Error finding container ffe89225ffc8480b499d233d345c24684fa7687c0aa66d9b03e4fa746ebb592c: Status 404 returned error can't find the container with id ffe89225ffc8480b499d233d345c24684fa7687c0aa66d9b03e4fa746ebb592c Jul 06 19:30:28.208736 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:28.208697 2565 generic.go:356] "Generic (PLEG): container finished" podID="8df2fa50-9067-4ed0-93bc-7743cacb7424" containerID="e853b3c28e346cb9252faf2560e4d32d25b1354cdc65a1f24ff7c43ca1d039fb" exitCode=0 Jul 06 19:30:28.208917 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:28.208788 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df2fa50-9067-4ed0-93bc-7743cacb7424","Type":"ContainerDied","Data":"e853b3c28e346cb9252faf2560e4d32d25b1354cdc65a1f24ff7c43ca1d039fb"} Jul 06 19:30:28.208917 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:28.208872 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df2fa50-9067-4ed0-93bc-7743cacb7424","Type":"ContainerStarted","Data":"ffe89225ffc8480b499d233d345c24684fa7687c0aa66d9b03e4fa746ebb592c"} Jul 06 19:30:29.206887 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:29.206850 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b428a9c7-6b1b-42ff-974d-7dd72b99bb22" path="/var/lib/kubelet/pods/b428a9c7-6b1b-42ff-974d-7dd72b99bb22/volumes" Jul 06 19:30:29.217166 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:29.217123 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df2fa50-9067-4ed0-93bc-7743cacb7424","Type":"ContainerStarted","Data":"a61b093e515440132286a5fbcf61f39b5b147ac9129397a08ebe31c85e609752"} Jul 06 19:30:29.217414 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:29.217233 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df2fa50-9067-4ed0-93bc-7743cacb7424","Type":"ContainerStarted","Data":"eb1c6fa076e8139202946104bed40edc26e197229099f564d06779fddaf37991"} Jul 06 19:30:29.217414 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:29.217269 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df2fa50-9067-4ed0-93bc-7743cacb7424","Type":"ContainerStarted","Data":"42a744a881465538cc766c3d48de12f525dc744b693dad15872c9e7c0786042d"} Jul 06 19:30:29.217414 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:29.217310 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df2fa50-9067-4ed0-93bc-7743cacb7424","Type":"ContainerStarted","Data":"3abcd7f2be8f860d080dbd3770eb5c08f580ad8d457aa6d967513f8ae1b78b50"} Jul 06 19:30:29.217414 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:29.217346 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df2fa50-9067-4ed0-93bc-7743cacb7424","Type":"ContainerStarted","Data":"a0432f43779369f4ac49d319d9f046255fb02b1706e8ec65237eda47a20e649b"} Jul 06 19:30:29.217414 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:29.217382 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df2fa50-9067-4ed0-93bc-7743cacb7424","Type":"ContainerStarted","Data":"2c100961ad62f4f774a94cfa87e260659f36962d4d70b6b6b1811753c1e0c4f8"} Jul 06 19:30:29.255458 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:29.255393 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.236595147 podStartE2EDuration="2.255372451s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.777304ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:29.254776665 +0000 UTC m=+260.629232619" watchObservedRunningTime="2026-07-06 19:30:29.255372451 +0000 UTC m=+260.629828391" Jul 06 19:30:32.606410 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:30:32.606326 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:31:09.065284 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:09.065254 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:31:09.070104 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:09.070078 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:31:09.071775 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:09.071758 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:31:09.076097 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:09.076078 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:31:09.076931 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:09.076916 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:31:27.606689 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:27.606650 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:31:27.622103 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:27.622076 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:31:28.461449 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:28.461423 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:31:53.149845 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.149808 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz"] Jul 06 19:31:53.153342 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.153326 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" Jul 06 19:31:53.158630 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.158600 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 19:31:53.159841 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.159816 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-tvrdn\"" Jul 06 19:31:53.159926 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.159819 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 19:31:53.175902 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.175872 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz"] Jul 06 19:31:53.256679 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.256641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e6d943d-fa09-4faf-9213-e02453d768ad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xzcsz\" (UID: \"0e6d943d-fa09-4faf-9213-e02453d768ad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" Jul 06 19:31:53.256679 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.256683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhgk2\" (UniqueName: \"kubernetes.io/projected/0e6d943d-fa09-4faf-9213-e02453d768ad-kube-api-access-mhgk2\") pod \"cert-manager-cainjector-559cdd8c87-xzcsz\" (UID: \"0e6d943d-fa09-4faf-9213-e02453d768ad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" Jul 06 19:31:53.358163 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.358120 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e6d943d-fa09-4faf-9213-e02453d768ad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xzcsz\" (UID: \"0e6d943d-fa09-4faf-9213-e02453d768ad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" Jul 06 19:31:53.358163 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.358166 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhgk2\" (UniqueName: \"kubernetes.io/projected/0e6d943d-fa09-4faf-9213-e02453d768ad-kube-api-access-mhgk2\") pod \"cert-manager-cainjector-559cdd8c87-xzcsz\" (UID: \"0e6d943d-fa09-4faf-9213-e02453d768ad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" Jul 06 19:31:53.389255 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.389202 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhgk2\" (UniqueName: \"kubernetes.io/projected/0e6d943d-fa09-4faf-9213-e02453d768ad-kube-api-access-mhgk2\") pod \"cert-manager-cainjector-559cdd8c87-xzcsz\" (UID: \"0e6d943d-fa09-4faf-9213-e02453d768ad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" Jul 06 19:31:53.390109 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.390087 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e6d943d-fa09-4faf-9213-e02453d768ad-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xzcsz\" (UID: \"0e6d943d-fa09-4faf-9213-e02453d768ad\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" Jul 06 19:31:53.472293 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.472198 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" Jul 06 19:31:53.607846 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.607820 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz"] Jul 06 19:31:53.610520 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:31:53.610486 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e6d943d_fa09_4faf_9213_e02453d768ad.slice/crio-e7c439fd007126be3c4a2839d4d9778b96428bcca46a11dc3912eadd86eca3f1 WatchSource:0}: Error finding container e7c439fd007126be3c4a2839d4d9778b96428bcca46a11dc3912eadd86eca3f1: Status 404 returned error can't find the container with id e7c439fd007126be3c4a2839d4d9778b96428bcca46a11dc3912eadd86eca3f1 Jul 06 19:31:53.612406 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:53.612391 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:31:54.545738 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:54.545661 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" event={"ID":"0e6d943d-fa09-4faf-9213-e02453d768ad","Type":"ContainerStarted","Data":"e7c439fd007126be3c4a2839d4d9778b96428bcca46a11dc3912eadd86eca3f1"} Jul 06 19:31:56.557613 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:56.557578 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" event={"ID":"0e6d943d-fa09-4faf-9213-e02453d768ad","Type":"ContainerStarted","Data":"dfccfd69efb7e746fe7188d60ae0c6863a78aefcc1eef9c949cdd7c2cc80fd19"} Jul 06 19:31:56.586461 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:31:56.586393 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xzcsz" podStartSLOduration=0.711470333 podStartE2EDuration="3.586375064s" totalImagesPullingTime="2.874904731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:56.584378284 +0000 UTC m=+347.958834226" watchObservedRunningTime="2026-07-06 19:31:56.586375064 +0000 UTC m=+347.960831007" Jul 06 19:32:22.661962 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.661885 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx"] Jul 06 19:32:22.665225 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.665190 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.671318 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.671249 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:32:22.671318 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.671292 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7kjrq\"" Jul 06 19:32:22.671525 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.671297 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:32:22.671525 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.671306 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:32:22.671525 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.671395 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:32:22.693228 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.693176 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx"] Jul 06 19:32:22.806624 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.806581 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d43247a6-92b9-450f-a3cc-fdcdeec48774-tmp\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.806816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.806689 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d43247a6-92b9-450f-a3cc-fdcdeec48774-manifests\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.806816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.806738 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d43247a6-92b9-450f-a3cc-fdcdeec48774-webhook-cert\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.806816 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.806768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d43247a6-92b9-450f-a3cc-fdcdeec48774-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.806935 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.806817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8f4t\" (UniqueName: \"kubernetes.io/projected/d43247a6-92b9-450f-a3cc-fdcdeec48774-kube-api-access-n8f4t\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.907456 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.907412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d43247a6-92b9-450f-a3cc-fdcdeec48774-webhook-cert\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.907456 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.907461 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d43247a6-92b9-450f-a3cc-fdcdeec48774-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.907700 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.907570 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8f4t\" (UniqueName: \"kubernetes.io/projected/d43247a6-92b9-450f-a3cc-fdcdeec48774-kube-api-access-n8f4t\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.907700 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.907636 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d43247a6-92b9-450f-a3cc-fdcdeec48774-tmp\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.907792 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.907708 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d43247a6-92b9-450f-a3cc-fdcdeec48774-manifests\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.908102 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.908072 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d43247a6-92b9-450f-a3cc-fdcdeec48774-manifests\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.909808 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.909779 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d43247a6-92b9-450f-a3cc-fdcdeec48774-tmp\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.910053 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.910038 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d43247a6-92b9-450f-a3cc-fdcdeec48774-webhook-cert\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.910111 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.910089 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d43247a6-92b9-450f-a3cc-fdcdeec48774-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.918893 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.918836 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8f4t\" (UniqueName: \"kubernetes.io/projected/d43247a6-92b9-450f-a3cc-fdcdeec48774-kube-api-access-n8f4t\") pod \"opendatahub-operator-controller-manager-5586496c87-bnjhx\" (UID: \"d43247a6-92b9-450f-a3cc-fdcdeec48774\") " pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:22.975859 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:22.975819 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:23.129967 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:23.129925 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx"] Jul 06 19:32:23.132236 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:32:23.132189 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43247a6_92b9_450f_a3cc_fdcdeec48774.slice/crio-dccd182688cf7fc9857a002717795b76bc37e37f560b8e7513b924529eba1ddd WatchSource:0}: Error finding container dccd182688cf7fc9857a002717795b76bc37e37f560b8e7513b924529eba1ddd: Status 404 returned error can't find the container with id dccd182688cf7fc9857a002717795b76bc37e37f560b8e7513b924529eba1ddd Jul 06 19:32:23.668595 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:23.668550 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" event={"ID":"d43247a6-92b9-450f-a3cc-fdcdeec48774","Type":"ContainerStarted","Data":"dccd182688cf7fc9857a002717795b76bc37e37f560b8e7513b924529eba1ddd"} Jul 06 19:32:26.685061 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:26.685022 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" event={"ID":"d43247a6-92b9-450f-a3cc-fdcdeec48774","Type":"ContainerStarted","Data":"1c5427dc121f691230d703ac9dec9fd1592c8eb0a5f2a3d6626d25d72bb58b75"} Jul 06 19:32:27.690396 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:27.690341 2565 generic.go:356] "Generic (PLEG): container finished" podID="d43247a6-92b9-450f-a3cc-fdcdeec48774" containerID="1c5427dc121f691230d703ac9dec9fd1592c8eb0a5f2a3d6626d25d72bb58b75" exitCode=0 Jul 06 19:32:27.690823 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:27.690436 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" event={"ID":"d43247a6-92b9-450f-a3cc-fdcdeec48774","Type":"ContainerDied","Data":"1c5427dc121f691230d703ac9dec9fd1592c8eb0a5f2a3d6626d25d72bb58b75"} Jul 06 19:32:28.696669 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:28.696633 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" event={"ID":"d43247a6-92b9-450f-a3cc-fdcdeec48774","Type":"ContainerStarted","Data":"7870fa6ddbd68474df3b90cbf3dc76fac53d574be90f979aa1861f3f44430115"} Jul 06 19:32:28.697134 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:28.696740 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:39.703144 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.703112 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:32:39.730192 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.730075 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" podStartSLOduration=13.145561261 podStartE2EDuration="17.730055868s" totalImagesPullingTime="3.456518519s" totalInitContainerRuntime="1.127976088s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:28.73538893 +0000 UTC m=+380.109844873" watchObservedRunningTime="2026-07-06 19:32:39.730055868 +0000 UTC m=+391.104511808" Jul 06 19:32:39.831267 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.831227 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-94489db4c-rnglb"] Jul 06 19:32:39.834463 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.834442 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.837791 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.837767 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:32:39.837929 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.837789 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 19:32:39.837929 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.837881 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 19:32:39.837929 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.837886 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 19:32:39.837929 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.837897 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-znr6f\"" Jul 06 19:32:39.838200 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.838158 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:32:39.850744 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.850721 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-94489db4c-rnglb"] Jul 06 19:32:39.852516 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.852496 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f557ac52-d837-47c1-931b-8b48f7462122-metrics-cert\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.852618 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.852535 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f557ac52-d837-47c1-931b-8b48f7462122-cert\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.852618 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.852571 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4wmp\" (UniqueName: \"kubernetes.io/projected/f557ac52-d837-47c1-931b-8b48f7462122-kube-api-access-z4wmp\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.852688 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.852632 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f557ac52-d837-47c1-931b-8b48f7462122-manager-config\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.953800 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.953712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4wmp\" (UniqueName: \"kubernetes.io/projected/f557ac52-d837-47c1-931b-8b48f7462122-kube-api-access-z4wmp\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.953800 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.953754 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f557ac52-d837-47c1-931b-8b48f7462122-manager-config\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.953800 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.953803 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f557ac52-d837-47c1-931b-8b48f7462122-metrics-cert\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.954090 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.953835 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f557ac52-d837-47c1-931b-8b48f7462122-cert\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.954540 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.954515 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f557ac52-d837-47c1-931b-8b48f7462122-manager-config\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.956370 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.956349 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f557ac52-d837-47c1-931b-8b48f7462122-metrics-cert\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.956654 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.956632 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f557ac52-d837-47c1-931b-8b48f7462122-cert\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:39.964194 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:39.964172 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4wmp\" (UniqueName: \"kubernetes.io/projected/f557ac52-d837-47c1-931b-8b48f7462122-kube-api-access-z4wmp\") pod \"lws-controller-manager-94489db4c-rnglb\" (UID: \"f557ac52-d837-47c1-931b-8b48f7462122\") " pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:40.143485 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:40.143445 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:40.276688 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:40.276659 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-94489db4c-rnglb"] Jul 06 19:32:40.278986 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:32:40.278954 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf557ac52_d837_47c1_931b_8b48f7462122.slice/crio-ab4a336fd2b5d8a789d3aaaac7625eeb29c433da65f9d0aef10ec99d118e5d3e WatchSource:0}: Error finding container ab4a336fd2b5d8a789d3aaaac7625eeb29c433da65f9d0aef10ec99d118e5d3e: Status 404 returned error can't find the container with id ab4a336fd2b5d8a789d3aaaac7625eeb29c433da65f9d0aef10ec99d118e5d3e Jul 06 19:32:40.746747 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:40.746709 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" event={"ID":"f557ac52-d837-47c1-931b-8b48f7462122","Type":"ContainerStarted","Data":"ab4a336fd2b5d8a789d3aaaac7625eeb29c433da65f9d0aef10ec99d118e5d3e"} Jul 06 19:32:42.757753 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:42.757711 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" event={"ID":"f557ac52-d837-47c1-931b-8b48f7462122","Type":"ContainerStarted","Data":"337da6dd5ca244a5d0c56d42afe4289894671acb5bc6b0389ac94555937f7d6e"} Jul 06 19:32:42.758128 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:42.757864 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:42.841268 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:42.841189 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" podStartSLOduration=1.52587386 podStartE2EDuration="3.841171379s" totalImagesPullingTime="2.315297519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:42.838175433 +0000 UTC m=+394.212631372" watchObservedRunningTime="2026-07-06 19:32:42.841171379 +0000 UTC m=+394.215627318" Jul 06 19:32:50.313923 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.313890 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr"] Jul 06 19:32:50.317321 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.317304 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.322852 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.322826 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 19:32:50.322996 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.322898 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:32:50.324756 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.324738 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-hm8fv\"" Jul 06 19:32:50.325026 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.325010 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:32:50.336889 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.336864 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr"] Jul 06 19:32:50.342505 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342480 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.342612 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342515 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.342612 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.342675 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342603 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.342716 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342673 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plx75\" (UniqueName: \"kubernetes.io/projected/ab90aa20-9137-4db0-9de5-31320ef227a1-kube-api-access-plx75\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.342752 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342712 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab90aa20-9137-4db0-9de5-31320ef227a1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.342782 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342753 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.342849 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342833 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.342916 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.342903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444137 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444099 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444137 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444137 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444395 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444155 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444395 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444173 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plx75\" (UniqueName: \"kubernetes.io/projected/ab90aa20-9137-4db0-9de5-31320ef227a1-kube-api-access-plx75\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444395 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444201 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab90aa20-9137-4db0-9de5-31320ef227a1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444395 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444395 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444395 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444366 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444703 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444398 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444703 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444575 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444703 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444639 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444853 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444697 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.444853 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.444732 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.445150 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.445130 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab90aa20-9137-4db0-9de5-31320ef227a1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.446661 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.446639 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.446743 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.446718 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.460861 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.460808 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab90aa20-9137-4db0-9de5-31320ef227a1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.461684 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.461660 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plx75\" (UniqueName: \"kubernetes.io/projected/ab90aa20-9137-4db0-9de5-31320ef227a1-kube-api-access-plx75\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjpfr\" (UID: \"ab90aa20-9137-4db0-9de5-31320ef227a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.627565 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.627527 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:50.796090 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:50.796059 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr"] Jul 06 19:32:50.799025 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:32:50.798996 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab90aa20_9137_4db0_9de5_31320ef227a1.slice/crio-714465151cd3a6092cb2e7c94ff57803947dcee72474f5f3829182283616b013 WatchSource:0}: Error finding container 714465151cd3a6092cb2e7c94ff57803947dcee72474f5f3829182283616b013: Status 404 returned error can't find the container with id 714465151cd3a6092cb2e7c94ff57803947dcee72474f5f3829182283616b013 Jul 06 19:32:51.795378 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:51.795340 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" event={"ID":"ab90aa20-9137-4db0-9de5-31320ef227a1","Type":"ContainerStarted","Data":"714465151cd3a6092cb2e7c94ff57803947dcee72474f5f3829182283616b013"} Jul 06 19:32:53.554767 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.554732 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl"] Jul 06 19:32:53.562406 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.562352 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.565485 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.565455 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 19:32:53.565883 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.565864 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 19:32:53.566084 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.566004 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 19:32:53.566084 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.566058 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-j9h48\"" Jul 06 19:32:53.576966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.576936 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl"] Jul 06 19:32:53.677090 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.677052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.677307 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.677101 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.677307 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.677161 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.677307 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.677257 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfntx\" (UniqueName: \"kubernetes.io/projected/aff7b4b2-5696-41d6-896c-4925c2e3f554-kube-api-access-lfntx\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.677307 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.677282 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff7b4b2-5696-41d6-896c-4925c2e3f554-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.677527 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.677331 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.677527 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.677364 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.763563 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.763533 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-94489db4c-rnglb" Jul 06 19:32:53.778323 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.778286 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.778323 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.778324 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.778577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.778345 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.778577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.778384 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfntx\" (UniqueName: \"kubernetes.io/projected/aff7b4b2-5696-41d6-896c-4925c2e3f554-kube-api-access-lfntx\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.778577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.778408 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff7b4b2-5696-41d6-896c-4925c2e3f554-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.778577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.778455 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.778577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.778481 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.779121 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.779093 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.780885 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.780855 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.780979 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.780950 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.781262 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.781238 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff7b4b2-5696-41d6-896c-4925c2e3f554-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.781383 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.781268 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.790469 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.790435 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff7b4b2-5696-41d6-896c-4925c2e3f554-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.790747 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.790727 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfntx\" (UniqueName: \"kubernetes.io/projected/aff7b4b2-5696-41d6-896c-4925c2e3f554-kube-api-access-lfntx\") pod \"istiod-openshift-gateway-94bb8fbfd-hcnsl\" (UID: \"aff7b4b2-5696-41d6-896c-4925c2e3f554\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:53.813521 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.813473 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:32:53.813646 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.813570 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:32:53.813646 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.813609 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:32:53.875151 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:53.875118 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:54.040911 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:54.040885 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl"] Jul 06 19:32:54.043489 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:32:54.043458 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff7b4b2_5696_41d6_896c_4925c2e3f554.slice/crio-8e910622176eee6708d18345c1ae74be8de3f5bc2e5c328d482579afdc697c90 WatchSource:0}: Error finding container 8e910622176eee6708d18345c1ae74be8de3f5bc2e5c328d482579afdc697c90: Status 404 returned error can't find the container with id 8e910622176eee6708d18345c1ae74be8de3f5bc2e5c328d482579afdc697c90 Jul 06 19:32:54.811910 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:54.811871 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" event={"ID":"ab90aa20-9137-4db0-9de5-31320ef227a1","Type":"ContainerStarted","Data":"4706c5ade013ea474d0faa252ce78f0ff01ffb83ca63fe6978565936aeb79971"} Jul 06 19:32:54.813115 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:54.813090 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" event={"ID":"aff7b4b2-5696-41d6-896c-4925c2e3f554","Type":"ContainerStarted","Data":"8e910622176eee6708d18345c1ae74be8de3f5bc2e5c328d482579afdc697c90"} Jul 06 19:32:54.856947 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:54.856889 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" podStartSLOduration=1.8448227099999999 podStartE2EDuration="4.856868644s" totalImagesPullingTime="3.012045934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:54.856000574 +0000 UTC m=+406.230456513" watchObservedRunningTime="2026-07-06 19:32:54.856868644 +0000 UTC m=+406.231324588" Jul 06 19:32:55.628495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:55.628458 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:55.633949 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:55.633916 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:55.817995 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:55.817960 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:55.819139 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:55.819110 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjpfr" Jul 06 19:32:56.251324 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:56.251291 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:32:56.251401 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:56.251355 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:32:56.825564 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:56.825530 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" event={"ID":"aff7b4b2-5696-41d6-896c-4925c2e3f554","Type":"ContainerStarted","Data":"da43f35787d24ab8a6b257b548807b9e496db2445b0c7ec00fdfaa8dfb2420c1"} Jul 06 19:32:56.826259 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:56.826241 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:32:56.860478 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:56.860422 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" podStartSLOduration=1.655483297 podStartE2EDuration="3.860405161s" totalImagesPullingTime="2.204921864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:56.857860945 +0000 UTC m=+408.232316912" watchObservedRunningTime="2026-07-06 19:32:56.860405161 +0000 UTC m=+408.234861101" Jul 06 19:32:57.830169 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:32:57.830140 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hcnsl" Jul 06 19:35:08.669536 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.669463 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc"] Jul 06 19:35:08.671507 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.671492 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.675868 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.675842 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6qxb9\"" Jul 06 19:35:08.685696 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.685670 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc"] Jul 06 19:35:08.761400 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761364 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhhjc\" (UniqueName: \"kubernetes.io/projected/e91b4320-ae9e-4a37-b77e-dd445b836880-kube-api-access-mhhjc\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.761400 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761410 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.761614 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761431 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.761614 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.761614 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761541 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.761614 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.761614 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.761775 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761647 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.761775 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.761667 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e91b4320-ae9e-4a37-b77e-dd445b836880-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.862568 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862537 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.862568 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862570 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e91b4320-ae9e-4a37-b77e-dd445b836880-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.862777 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862601 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhhjc\" (UniqueName: \"kubernetes.io/projected/e91b4320-ae9e-4a37-b77e-dd445b836880-kube-api-access-mhhjc\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.862777 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.862777 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862765 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.862896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.862896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862861 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.862896 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.863043 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.862921 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.863043 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.863035 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.863170 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.863143 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.863364 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.863331 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.863475 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.863344 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.863475 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.863436 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e91b4320-ae9e-4a37-b77e-dd445b836880-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.865037 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.865019 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.865309 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.865294 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.874884 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.874850 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e91b4320-ae9e-4a37-b77e-dd445b836880-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.886200 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.886172 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhhjc\" (UniqueName: \"kubernetes.io/projected/e91b4320-ae9e-4a37-b77e-dd445b836880-kube-api-access-mhhjc\") pod \"maas-default-gateway-openshift-default-5896d4dd49-6pwvc\" (UID: \"e91b4320-ae9e-4a37-b77e-dd445b836880\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:08.982522 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:08.982435 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:09.124758 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:09.124722 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc"] Jul 06 19:35:09.127678 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:35:09.127651 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode91b4320_ae9e_4a37_b77e_dd445b836880.slice/crio-ebedd7d2c101aef07ed57ffdcda50acfec2a94b26f0fe867e4a9d99a50b2e941 WatchSource:0}: Error finding container ebedd7d2c101aef07ed57ffdcda50acfec2a94b26f0fe867e4a9d99a50b2e941: Status 404 returned error can't find the container with id ebedd7d2c101aef07ed57ffdcda50acfec2a94b26f0fe867e4a9d99a50b2e941 Jul 06 19:35:09.129575 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:09.129544 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:35:09.129644 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:09.129608 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:35:09.129644 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:09.129637 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:35:09.379751 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:09.379717 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" event={"ID":"e91b4320-ae9e-4a37-b77e-dd445b836880","Type":"ContainerStarted","Data":"89179f56a4c838191b3847797e23a2d722cbc8121e8bd7d27fcf0b966f4c9a86"} Jul 06 19:35:09.379929 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:09.379772 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" event={"ID":"e91b4320-ae9e-4a37-b77e-dd445b836880","Type":"ContainerStarted","Data":"ebedd7d2c101aef07ed57ffdcda50acfec2a94b26f0fe867e4a9d99a50b2e941"} Jul 06 19:35:09.406458 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:09.406410 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" podStartSLOduration=1.406386167 podStartE2EDuration="1.406386167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:09.404077259 +0000 UTC m=+540.778533218" watchObservedRunningTime="2026-07-06 19:35:09.406386167 +0000 UTC m=+540.780842108" Jul 06 19:35:09.982569 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:09.982535 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:10.988299 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:10.988271 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:11.388006 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:11.387977 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:35:11.389000 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:35:11.388975 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:36:09.093361 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:09.093333 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:36:09.094983 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:09.094961 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:36:09.098914 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:09.098895 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:36:09.100271 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:09.100254 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:36:23.746271 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.746237 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6c4669c765-jgq9q"] Jul 06 19:36:23.748390 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.748372 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.752743 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.752712 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-q8w7d\"" Jul 06 19:36:23.754047 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.754025 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 19:36:23.769468 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.769437 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6c4669c765-jgq9q"] Jul 06 19:36:23.821940 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.821898 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcg7q\" (UniqueName: \"kubernetes.io/projected/b1533c31-2ce6-4c55-bfd0-c993217553c8-kube-api-access-vcg7q\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.822108 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.821959 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1533c31-2ce6-4c55-bfd0-c993217553c8-maas-api-tls\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.822108 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.821986 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1533c31-2ce6-4c55-bfd0-c993217553c8-openshift-service-ca\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.923021 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.922979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcg7q\" (UniqueName: \"kubernetes.io/projected/b1533c31-2ce6-4c55-bfd0-c993217553c8-kube-api-access-vcg7q\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.923191 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.923034 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1533c31-2ce6-4c55-bfd0-c993217553c8-maas-api-tls\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.923191 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.923051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1533c31-2ce6-4c55-bfd0-c993217553c8-openshift-service-ca\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.923762 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.923742 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1533c31-2ce6-4c55-bfd0-c993217553c8-openshift-service-ca\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.925669 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.925646 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1533c31-2ce6-4c55-bfd0-c993217553c8-maas-api-tls\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:23.933742 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:23.933718 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcg7q\" (UniqueName: \"kubernetes.io/projected/b1533c31-2ce6-4c55-bfd0-c993217553c8-kube-api-access-vcg7q\") pod \"maas-api-6c4669c765-jgq9q\" (UID: \"b1533c31-2ce6-4c55-bfd0-c993217553c8\") " pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:24.058843 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.058748 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:24.408089 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.408064 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6c4669c765-jgq9q"] Jul 06 19:36:24.410600 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:36:24.410573 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1533c31_2ce6_4c55_bfd0_c993217553c8.slice/crio-b44c50072bb19e83e72c67d4b1220e8c588b317b7ce0ba77c88e320954db84a0 WatchSource:0}: Error finding container b44c50072bb19e83e72c67d4b1220e8c588b317b7ce0ba77c88e320954db84a0: Status 404 returned error can't find the container with id b44c50072bb19e83e72c67d4b1220e8c588b317b7ce0ba77c88e320954db84a0 Jul 06 19:36:24.526259 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.526222 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-hmg7z"] Jul 06 19:36:24.529034 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.529012 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:24.533810 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.533791 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dp8sp\"" Jul 06 19:36:24.534195 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.534172 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 19:36:24.547268 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.547207 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-hmg7z"] Jul 06 19:36:24.630695 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.630647 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzjh4\" (UniqueName: \"kubernetes.io/projected/c5bea8bb-2aba-4ba2-b5dd-663f11695f6d-kube-api-access-tzjh4\") pod \"payload-processing-58f86c7798-hmg7z\" (UID: \"c5bea8bb-2aba-4ba2-b5dd-663f11695f6d\") " pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:24.630888 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.630740 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bea8bb-2aba-4ba2-b5dd-663f11695f6d-plugins-config-volume\") pod \"payload-processing-58f86c7798-hmg7z\" (UID: \"c5bea8bb-2aba-4ba2-b5dd-663f11695f6d\") " pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:24.708942 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.708853 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6c4669c765-jgq9q" event={"ID":"b1533c31-2ce6-4c55-bfd0-c993217553c8","Type":"ContainerStarted","Data":"b44c50072bb19e83e72c67d4b1220e8c588b317b7ce0ba77c88e320954db84a0"} Jul 06 19:36:24.731341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.731296 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bea8bb-2aba-4ba2-b5dd-663f11695f6d-plugins-config-volume\") pod \"payload-processing-58f86c7798-hmg7z\" (UID: \"c5bea8bb-2aba-4ba2-b5dd-663f11695f6d\") " pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:24.731491 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.731382 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzjh4\" (UniqueName: \"kubernetes.io/projected/c5bea8bb-2aba-4ba2-b5dd-663f11695f6d-kube-api-access-tzjh4\") pod \"payload-processing-58f86c7798-hmg7z\" (UID: \"c5bea8bb-2aba-4ba2-b5dd-663f11695f6d\") " pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:24.731985 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.731966 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bea8bb-2aba-4ba2-b5dd-663f11695f6d-plugins-config-volume\") pod \"payload-processing-58f86c7798-hmg7z\" (UID: \"c5bea8bb-2aba-4ba2-b5dd-663f11695f6d\") " pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:24.744006 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.743978 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzjh4\" (UniqueName: \"kubernetes.io/projected/c5bea8bb-2aba-4ba2-b5dd-663f11695f6d-kube-api-access-tzjh4\") pod \"payload-processing-58f86c7798-hmg7z\" (UID: \"c5bea8bb-2aba-4ba2-b5dd-663f11695f6d\") " pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:24.839492 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:24.839437 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:25.000859 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:25.000770 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-hmg7z"] Jul 06 19:36:25.005095 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:36:25.005060 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5bea8bb_2aba_4ba2_b5dd_663f11695f6d.slice/crio-8c1a78f8fa58a5a83794666731710589903519e07e8bab1bd24be92430284697 WatchSource:0}: Error finding container 8c1a78f8fa58a5a83794666731710589903519e07e8bab1bd24be92430284697: Status 404 returned error can't find the container with id 8c1a78f8fa58a5a83794666731710589903519e07e8bab1bd24be92430284697 Jul 06 19:36:25.742236 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:25.741939 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" event={"ID":"c5bea8bb-2aba-4ba2-b5dd-663f11695f6d","Type":"ContainerStarted","Data":"8c1a78f8fa58a5a83794666731710589903519e07e8bab1bd24be92430284697"} Jul 06 19:36:27.753387 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:27.753239 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6c4669c765-jgq9q" event={"ID":"b1533c31-2ce6-4c55-bfd0-c993217553c8","Type":"ContainerStarted","Data":"02ba4e2f035ac1e5ebc002d965379af29827d3e887c2963be0f49e7b6183bb6b"} Jul 06 19:36:27.753387 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:27.753307 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:27.763369 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:27.754927 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" event={"ID":"c5bea8bb-2aba-4ba2-b5dd-663f11695f6d","Type":"ContainerStarted","Data":"46f5c774f3626ac9f5c6b4eb6e92285811411feac731a77e680e71cb9a4fdea1"} Jul 06 19:36:27.763369 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:27.754966 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:27.776619 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:27.776568 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6c4669c765-jgq9q" podStartSLOduration=2.341418309 podStartE2EDuration="4.776552048s" totalImagesPullingTime="2.435133739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:36:27.775799194 +0000 UTC m=+619.150255163" watchObservedRunningTime="2026-07-06 19:36:27.776552048 +0000 UTC m=+619.151008027" Jul 06 19:36:27.796968 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:27.796916 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" podStartSLOduration=1.9600954430000002 podStartE2EDuration="3.796900878s" totalImagesPullingTime="1.836805435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:36:27.795577754 +0000 UTC m=+619.170033694" watchObservedRunningTime="2026-07-06 19:36:27.796900878 +0000 UTC m=+619.171356818" Jul 06 19:36:32.777440 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:32.777409 2565 generic.go:356] "Generic (PLEG): container finished" podID="e91b4320-ae9e-4a37-b77e-dd445b836880" containerID="89179f56a4c838191b3847797e23a2d722cbc8121e8bd7d27fcf0b966f4c9a86" exitCode=0 Jul 06 19:36:32.777862 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:32.777478 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" event={"ID":"e91b4320-ae9e-4a37-b77e-dd445b836880","Type":"ContainerDied","Data":"89179f56a4c838191b3847797e23a2d722cbc8121e8bd7d27fcf0b966f4c9a86"} Jul 06 19:36:32.777862 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:32.777857 2565 scope.go:117] "RemoveContainer" containerID="89179f56a4c838191b3847797e23a2d722cbc8121e8bd7d27fcf0b966f4c9a86" Jul 06 19:36:32.778858 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:32.778829 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:36:32.778931 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:32.778914 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:36:32.778981 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:32.778965 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:36:33.765409 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:33.765379 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6c4669c765-jgq9q" Jul 06 19:36:33.784113 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:33.784077 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" event={"ID":"e91b4320-ae9e-4a37-b77e-dd445b836880","Type":"ContainerStarted","Data":"dfbadc1a68afc36597e65b000197892cfa74285e37a7a9d1b43b2ed47187cd6a"} Jul 06 19:36:33.982867 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:33.982830 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:36:33.983178 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:33.983149 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-6pwvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 06 19:36:33.983277 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:33.983248 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" podUID="e91b4320-ae9e-4a37-b77e-dd445b836880" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 06 19:36:34.983454 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:34.983415 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-6pwvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 06 19:36:34.983923 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:34.983492 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" podUID="e91b4320-ae9e-4a37-b77e-dd445b836880" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 06 19:36:35.983817 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:35.983776 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-6pwvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 06 19:36:35.984189 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:35.983847 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" podUID="e91b4320-ae9e-4a37-b77e-dd445b836880" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 06 19:36:36.982954 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:36.982916 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-6pwvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 06 19:36:36.983139 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:36.982975 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" podUID="e91b4320-ae9e-4a37-b77e-dd445b836880" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 06 19:36:37.982978 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:37.982940 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-6pwvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 06 19:36:37.983406 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:37.982997 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" podUID="e91b4320-ae9e-4a37-b77e-dd445b836880" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 06 19:36:38.761886 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:38.761852 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-hmg7z" Jul 06 19:36:38.986768 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:38.986733 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:36:38.987188 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:38.986991 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:36:38.987638 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:36:38.987618 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-6pwvc" Jul 06 19:37:02.822044 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.822012 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n"] Jul 06 19:37:02.826553 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.826531 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.829341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.829319 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:37:02.830674 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.830650 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:37:02.830674 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.830667 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 19:37:02.830971 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.830956 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cqnv7\"" Jul 06 19:37:02.838259 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.838234 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n"] Jul 06 19:37:02.873322 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.873294 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.873443 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.873329 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5kkn\" (UniqueName: \"kubernetes.io/projected/d5478f16-fcbf-487a-ae7d-fc25120c37f8-kube-api-access-n5kkn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.873443 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.873365 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.873443 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.873394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5478f16-fcbf-487a-ae7d-fc25120c37f8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.873443 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.873421 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.873581 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.873453 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974298 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974266 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974489 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974305 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kkn\" (UniqueName: \"kubernetes.io/projected/d5478f16-fcbf-487a-ae7d-fc25120c37f8-kube-api-access-n5kkn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974489 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974332 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974489 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974353 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5478f16-fcbf-487a-ae7d-fc25120c37f8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974489 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974489 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974766 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974749 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974870 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974800 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.974909 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.974884 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.976703 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.976680 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5478f16-fcbf-487a-ae7d-fc25120c37f8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.976985 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.976963 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5478f16-fcbf-487a-ae7d-fc25120c37f8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:02.984831 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:02.984806 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5kkn\" (UniqueName: \"kubernetes.io/projected/d5478f16-fcbf-487a-ae7d-fc25120c37f8-kube-api-access-n5kkn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gqz7n\" (UID: \"d5478f16-fcbf-487a-ae7d-fc25120c37f8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:03.137425 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:03.137319 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:03.275829 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:03.275798 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n"] Jul 06 19:37:03.277837 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:37:03.277801 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5478f16_fcbf_487a_ae7d_fc25120c37f8.slice/crio-b1e7005a5b207f97d96cedca6a1b8539af3440e85f472876fdd87d41083cbbc8 WatchSource:0}: Error finding container b1e7005a5b207f97d96cedca6a1b8539af3440e85f472876fdd87d41083cbbc8: Status 404 returned error can't find the container with id b1e7005a5b207f97d96cedca6a1b8539af3440e85f472876fdd87d41083cbbc8 Jul 06 19:37:03.279104 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:03.279087 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:37:03.920439 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:03.920393 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" event={"ID":"d5478f16-fcbf-487a-ae7d-fc25120c37f8","Type":"ContainerStarted","Data":"b1e7005a5b207f97d96cedca6a1b8539af3440e85f472876fdd87d41083cbbc8"} Jul 06 19:37:04.925529 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:04.925493 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" event={"ID":"d5478f16-fcbf-487a-ae7d-fc25120c37f8","Type":"ContainerStarted","Data":"a52a47855ed34764543d4857197350189ba244f5549e4f935f7eee4d04531037"} Jul 06 19:37:04.953905 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:04.953858 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" podStartSLOduration=1.933423478 podStartE2EDuration="2.953843553s" totalImagesPullingTime="1.020420075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:37:04.952959808 +0000 UTC m=+656.327415748" watchObservedRunningTime="2026-07-06 19:37:04.953843553 +0000 UTC m=+656.328299493" Jul 06 19:37:13.138198 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:13.138156 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:13.138666 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:13.138245 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:13.151047 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:13.151016 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:13.978048 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:13.978021 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gqz7n" Jul 06 19:37:24.449717 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.449681 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm"] Jul 06 19:37:24.453098 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.453077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.456824 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.456803 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 19:37:24.464236 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.464203 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm"] Jul 06 19:37:24.570881 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.570848 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.570881 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.570880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62fd00c-4f13-4185-bd7c-eb85a44ec894-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.571097 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.570898 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nsh4\" (UniqueName: \"kubernetes.io/projected/b62fd00c-4f13-4185-bd7c-eb85a44ec894-kube-api-access-8nsh4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.571097 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.570998 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.571097 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.571063 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.571196 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.571101 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.671836 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.671792 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.672004 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.671951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.672048 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.672005 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.672048 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.672030 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.672048 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.672045 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62fd00c-4f13-4185-bd7c-eb85a44ec894-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.672171 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.672062 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nsh4\" (UniqueName: \"kubernetes.io/projected/b62fd00c-4f13-4185-bd7c-eb85a44ec894-kube-api-access-8nsh4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.672378 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.672357 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.672475 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.672398 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.672533 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.672471 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.674032 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.674012 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b62fd00c-4f13-4185-bd7c-eb85a44ec894-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.674508 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.674489 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b62fd00c-4f13-4185-bd7c-eb85a44ec894-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.682447 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.682418 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nsh4\" (UniqueName: \"kubernetes.io/projected/b62fd00c-4f13-4185-bd7c-eb85a44ec894-kube-api-access-8nsh4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zk6bm\" (UID: \"b62fd00c-4f13-4185-bd7c-eb85a44ec894\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.764383 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.764293 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:24.906584 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:24.906560 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm"] Jul 06 19:37:24.908551 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:37:24.908525 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62fd00c_4f13_4185_bd7c_eb85a44ec894.slice/crio-af3d538ee5e65bf5a01883f6041e59ceb106a4a131464448f9a81b1c02f0acdd WatchSource:0}: Error finding container af3d538ee5e65bf5a01883f6041e59ceb106a4a131464448f9a81b1c02f0acdd: Status 404 returned error can't find the container with id af3d538ee5e65bf5a01883f6041e59ceb106a4a131464448f9a81b1c02f0acdd Jul 06 19:37:25.019868 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:25.019777 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" event={"ID":"b62fd00c-4f13-4185-bd7c-eb85a44ec894","Type":"ContainerStarted","Data":"af3d538ee5e65bf5a01883f6041e59ceb106a4a131464448f9a81b1c02f0acdd"} Jul 06 19:37:26.025665 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:26.025628 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" event={"ID":"b62fd00c-4f13-4185-bd7c-eb85a44ec894","Type":"ContainerStarted","Data":"08b91c064dea9c29a02736ca9f880e941aa7e6c87b764eb9fb800f79060388d4"} Jul 06 19:37:26.051756 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:26.051702 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" podStartSLOduration=1.843258259 podStartE2EDuration="2.051687176s" totalImagesPullingTime="208.428917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:37:26.051630363 +0000 UTC m=+677.426086305" watchObservedRunningTime="2026-07-06 19:37:26.051687176 +0000 UTC m=+677.426143116" Jul 06 19:37:34.764487 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:34.764447 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:34.765010 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:34.764518 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:34.777349 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:34.777309 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:37:35.078354 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:37:35.078320 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zk6bm" Jul 06 19:41:09.123588 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:41:09.123509 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:41:09.123588 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:41:09.123559 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:41:09.129572 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:41:09.129549 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:41:09.129708 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:41:09.129611 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:45:49.327301 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:45:49.327265 2565 generic.go:356] "Generic (PLEG): container finished" podID="d43247a6-92b9-450f-a3cc-fdcdeec48774" containerID="7870fa6ddbd68474df3b90cbf3dc76fac53d574be90f979aa1861f3f44430115" exitCode=1 Jul 06 19:45:49.327712 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:45:49.327323 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" event={"ID":"d43247a6-92b9-450f-a3cc-fdcdeec48774","Type":"ContainerDied","Data":"7870fa6ddbd68474df3b90cbf3dc76fac53d574be90f979aa1861f3f44430115"} Jul 06 19:45:49.327712 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:45:49.327682 2565 scope.go:117] "RemoveContainer" containerID="7870fa6ddbd68474df3b90cbf3dc76fac53d574be90f979aa1861f3f44430115" Jul 06 19:45:49.327970 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:45:49.327958 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:45:49.701777 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:45:49.701692 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:45:50.334591 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:45:50.334554 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" event={"ID":"d43247a6-92b9-450f-a3cc-fdcdeec48774","Type":"ContainerStarted","Data":"bc5eb23c00f03682e153d1c36ae6b18c65a54b9a11867938285a1b49ccc8631e"} Jul 06 19:45:50.334983 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:45:50.334685 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:46:01.340860 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:46:01.340822 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5586496c87-bnjhx" Jul 06 19:46:09.155106 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:46:09.155068 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:46:09.156077 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:46:09.156055 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:46:09.160622 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:46:09.160599 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:46:09.161537 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:46:09.161518 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:51:09.182806 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:51:09.182773 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:51:09.184322 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:51:09.184299 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:51:09.188102 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:51:09.188081 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:51:09.189428 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:51:09.189411 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:52:17.237772 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.237733 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6"] Jul 06 19:52:17.240898 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.240874 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.243769 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.243745 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-6acba6da-serving-cert\"" Jul 06 19:52:17.254733 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.254705 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6"] Jul 06 19:52:17.290076 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.290041 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m8z9\" (UniqueName: \"kubernetes.io/projected/1e973ca9-9826-4408-a5a0-77296839e7a2-kube-api-access-4m8z9\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.290282 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.290104 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.290282 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.290159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e973ca9-9826-4408-a5a0-77296839e7a2-openshift-service-ca\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.390692 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.390650 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m8z9\" (UniqueName: \"kubernetes.io/projected/1e973ca9-9826-4408-a5a0-77296839e7a2-kube-api-access-4m8z9\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.390692 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.390693 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.390961 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.390724 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e973ca9-9826-4408-a5a0-77296839e7a2-openshift-service-ca\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.390961 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:52:17.390825 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-6acba6da-serving-cert: secret "maas-api-e2e-ait-6acba6da-serving-cert" not found Jul 06 19:52:17.390961 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:52:17.390918 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls podName:1e973ca9-9826-4408-a5a0-77296839e7a2 nodeName:}" failed. No retries permitted until 2026-07-06 19:52:17.890896843 +0000 UTC m=+1569.265352780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls") pod "maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" (UID: "1e973ca9-9826-4408-a5a0-77296839e7a2") : secret "maas-api-e2e-ait-6acba6da-serving-cert" not found Jul 06 19:52:17.391403 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.391386 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e973ca9-9826-4408-a5a0-77296839e7a2-openshift-service-ca\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.402140 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.402109 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m8z9\" (UniqueName: \"kubernetes.io/projected/1e973ca9-9826-4408-a5a0-77296839e7a2-kube-api-access-4m8z9\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.895669 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.895635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:17.898326 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:17.898287 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls\") pod \"maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:18.151699 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:18.151593 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:18.287055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:18.287027 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6"] Jul 06 19:52:18.289198 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:52:18.289169 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e973ca9_9826_4408_a5a0_77296839e7a2.slice/crio-14810a8198deb6a25046d49c832eb1bf99ed6bf00a6fb607226daf2f5c7f5076 WatchSource:0}: Error finding container 14810a8198deb6a25046d49c832eb1bf99ed6bf00a6fb607226daf2f5c7f5076: Status 404 returned error can't find the container with id 14810a8198deb6a25046d49c832eb1bf99ed6bf00a6fb607226daf2f5c7f5076 Jul 06 19:52:18.290558 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:18.290540 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:52:19.119798 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:19.119755 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" event={"ID":"1e973ca9-9826-4408-a5a0-77296839e7a2","Type":"ContainerStarted","Data":"91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844"} Jul 06 19:52:19.119980 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:19.119847 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" event={"ID":"1e973ca9-9826-4408-a5a0-77296839e7a2","Type":"ContainerStarted","Data":"14810a8198deb6a25046d49c832eb1bf99ed6bf00a6fb607226daf2f5c7f5076"} Jul 06 19:52:19.119980 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:19.119876 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:19.144520 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:19.144469 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" podStartSLOduration=1.696321543 podStartE2EDuration="2.144453244s" totalImagesPullingTime="448.131701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:52:19.142763948 +0000 UTC m=+1570.517219888" watchObservedRunningTime="2026-07-06 19:52:19.144453244 +0000 UTC m=+1570.518909240" Jul 06 19:52:22.568575 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.568543 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6"] Jul 06 19:52:22.569055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.568733 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" podUID="1e973ca9-9826-4408-a5a0-77296839e7a2" containerName="maas-api" containerID="cri-o://91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844" gracePeriod=30 Jul 06 19:52:22.812576 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.812548 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:22.834125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.834030 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e973ca9-9826-4408-a5a0-77296839e7a2-openshift-service-ca\") pod \"1e973ca9-9826-4408-a5a0-77296839e7a2\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " Jul 06 19:52:22.834125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.834081 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls\") pod \"1e973ca9-9826-4408-a5a0-77296839e7a2\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " Jul 06 19:52:22.834384 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.834163 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4m8z9\" (UniqueName: \"kubernetes.io/projected/1e973ca9-9826-4408-a5a0-77296839e7a2-kube-api-access-4m8z9\") pod \"1e973ca9-9826-4408-a5a0-77296839e7a2\" (UID: \"1e973ca9-9826-4408-a5a0-77296839e7a2\") " Jul 06 19:52:22.834484 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.834446 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e973ca9-9826-4408-a5a0-77296839e7a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e973ca9-9826-4408-a5a0-77296839e7a2" (UID: "1e973ca9-9826-4408-a5a0-77296839e7a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:52:22.836616 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.836583 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e973ca9-9826-4408-a5a0-77296839e7a2-kube-api-access-4m8z9" (OuterVolumeSpecName: "kube-api-access-4m8z9") pod "1e973ca9-9826-4408-a5a0-77296839e7a2" (UID: "1e973ca9-9826-4408-a5a0-77296839e7a2"). InnerVolumeSpecName "kube-api-access-4m8z9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:52:22.836745 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.836641 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e973ca9-9826-4408-a5a0-77296839e7a2" (UID: "1e973ca9-9826-4408-a5a0-77296839e7a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:52:22.935449 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.935411 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m8z9\" (UniqueName: \"kubernetes.io/projected/1e973ca9-9826-4408-a5a0-77296839e7a2-kube-api-access-4m8z9\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:22.935449 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.935444 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e973ca9-9826-4408-a5a0-77296839e7a2-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:22.935449 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:22.935453 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e973ca9-9826-4408-a5a0-77296839e7a2-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:23.139893 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.139798 2565 generic.go:356] "Generic (PLEG): container finished" podID="1e973ca9-9826-4408-a5a0-77296839e7a2" containerID="91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844" exitCode=0 Jul 06 19:52:23.139893 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.139846 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" Jul 06 19:52:23.139893 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.139846 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" event={"ID":"1e973ca9-9826-4408-a5a0-77296839e7a2","Type":"ContainerDied","Data":"91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844"} Jul 06 19:52:23.140161 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.139898 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6" event={"ID":"1e973ca9-9826-4408-a5a0-77296839e7a2","Type":"ContainerDied","Data":"14810a8198deb6a25046d49c832eb1bf99ed6bf00a6fb607226daf2f5c7f5076"} Jul 06 19:52:23.140161 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.139922 2565 scope.go:117] "RemoveContainer" containerID="91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844" Jul 06 19:52:23.150591 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.150571 2565 scope.go:117] "RemoveContainer" containerID="91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844" Jul 06 19:52:23.150915 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:52:23.150883 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844\": container with ID starting with 91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844 not found: ID does not exist" containerID="91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844" Jul 06 19:52:23.151004 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.150918 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844"} err="failed to get container status \"91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844\": rpc error: code = NotFound desc = could not find container \"91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844\": container with ID starting with 91f283646a812f3c0b9336d22ea7f898e31744b9f934ae73eceeee0cadd7b844 not found: ID does not exist" Jul 06 19:52:23.166574 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.166539 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6"] Jul 06 19:52:23.171567 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.171541 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-6acba6da-cd7b8cb9b-h7lz6"] Jul 06 19:52:23.209651 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:23.206719 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e973ca9-9826-4408-a5a0-77296839e7a2" path="/var/lib/kubelet/pods/1e973ca9-9826-4408-a5a0-77296839e7a2/volumes" Jul 06 19:52:29.251628 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.251579 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr"] Jul 06 19:52:29.252060 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.251952 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e973ca9-9826-4408-a5a0-77296839e7a2" containerName="maas-api" Jul 06 19:52:29.252060 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.251963 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e973ca9-9826-4408-a5a0-77296839e7a2" containerName="maas-api" Jul 06 19:52:29.252060 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.252021 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e973ca9-9826-4408-a5a0-77296839e7a2" containerName="maas-api" Jul 06 19:52:29.256563 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.256536 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.260414 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.260386 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a583ee7e-openshift-default-dockercfg-5hdsh\"" Jul 06 19:52:29.278129 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.278100 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr"] Jul 06 19:52:29.292776 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.292751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-certs\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.292904 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.292802 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cn5h\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-kube-api-access-5cn5h\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.292904 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.292834 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-socket\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.292904 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.292867 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-envoy\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.292904 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.292886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-istio-token\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.293125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.292911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-data\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.293125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.292970 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb4197de-6a46-4042-a40d-71df4110d5db-istiod-ca-cert\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.293125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.293029 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb4197de-6a46-4042-a40d-71df4110d5db-istio-podinfo\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.293125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.293099 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-credential-socket\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394374 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394341 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-data\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394382 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb4197de-6a46-4042-a40d-71df4110d5db-istiod-ca-cert\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb4197de-6a46-4042-a40d-71df4110d5db-istio-podinfo\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394455 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-credential-socket\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-certs\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394518 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cn5h\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-kube-api-access-5cn5h\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394577 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394551 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-socket\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394950 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-envoy\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394950 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394730 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-istio-token\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.394950 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394841 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-data\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.395107 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.394943 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-socket\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.395107 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.395072 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-credential-socket\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.395107 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.395090 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-certs\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.395450 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.395431 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb4197de-6a46-4042-a40d-71df4110d5db-istiod-ca-cert\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.396985 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.396954 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-envoy\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.397129 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.397114 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb4197de-6a46-4042-a40d-71df4110d5db-istio-podinfo\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.406427 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.406408 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-istio-token\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.406539 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.406481 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cn5h\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-kube-api-access-5cn5h\") pod \"e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.568728 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.568704 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:29.711748 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.711720 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr"] Jul 06 19:52:29.713386 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:52:29.713362 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4197de_6a46_4042_a40d_71df4110d5db.slice/crio-0a8ac8462ed8e48e37e307aafecb4879895339fddc7504c7e8ac84562c2d1afe WatchSource:0}: Error finding container 0a8ac8462ed8e48e37e307aafecb4879895339fddc7504c7e8ac84562c2d1afe: Status 404 returned error can't find the container with id 0a8ac8462ed8e48e37e307aafecb4879895339fddc7504c7e8ac84562c2d1afe Jul 06 19:52:29.716981 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.716579 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:52:29.716981 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.716731 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:52:29.716981 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:29.716824 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:52:30.174749 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:30.174711 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" event={"ID":"cb4197de-6a46-4042-a40d-71df4110d5db","Type":"ContainerStarted","Data":"d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2"} Jul 06 19:52:30.174958 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:30.174795 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" event={"ID":"cb4197de-6a46-4042-a40d-71df4110d5db","Type":"ContainerStarted","Data":"0a8ac8462ed8e48e37e307aafecb4879895339fddc7504c7e8ac84562c2d1afe"} Jul 06 19:52:30.198635 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:30.198579 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" podStartSLOduration=1.198561355 podStartE2EDuration="1.198561355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:52:30.197644157 +0000 UTC m=+1581.572100098" watchObservedRunningTime="2026-07-06 19:52:30.198561355 +0000 UTC m=+1581.573017299" Jul 06 19:52:30.569257 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:30.569204 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:30.574009 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:30.573981 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:31.179311 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:31.179281 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:31.180390 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:31.180371 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:41.225914 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:41.225821 2565 generic.go:356] "Generic (PLEG): container finished" podID="cb4197de-6a46-4042-a40d-71df4110d5db" containerID="d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2" exitCode=0 Jul 06 19:52:41.225914 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:41.225898 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" event={"ID":"cb4197de-6a46-4042-a40d-71df4110d5db","Type":"ContainerDied","Data":"d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2"} Jul 06 19:52:41.226373 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:41.226304 2565 scope.go:117] "RemoveContainer" containerID="d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2" Jul 06 19:52:41.227378 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:41.227347 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:52:41.227500 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:41.227414 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:52:41.227500 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:41.227445 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:52:41.701740 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:41.701702 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr"] Jul 06 19:52:42.232639 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:42.232599 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" event={"ID":"cb4197de-6a46-4042-a40d-71df4110d5db","Type":"ContainerStarted","Data":"1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da"} Jul 06 19:52:42.233145 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:42.232655 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" podUID="cb4197de-6a46-4042-a40d-71df4110d5db" containerName="istio-proxy" containerID="cri-o://1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da" gracePeriod=30 Jul 06 19:52:46.179630 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:46.179593 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:47.488014 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.487989 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:47.567392 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567363 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-istio-token\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567559 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567465 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb4197de-6a46-4042-a40d-71df4110d5db-istio-podinfo\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567559 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567515 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-credential-socket\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567559 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567556 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb4197de-6a46-4042-a40d-71df4110d5db-istiod-ca-cert\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567696 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567601 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-certs\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567696 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567633 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cn5h\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-kube-api-access-5cn5h\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567729 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-data\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567807 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567788 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-socket\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567913 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567811 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-envoy\") pod \"cb4197de-6a46-4042-a40d-71df4110d5db\" (UID: \"cb4197de-6a46-4042-a40d-71df4110d5db\") " Jul 06 19:52:47.567969 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.567897 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:52:47.568264 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.568051 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb4197de-6a46-4042-a40d-71df4110d5db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:52:47.568264 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.568159 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb4197de-6a46-4042-a40d-71df4110d5db-istiod-ca-cert\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:47.568264 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.568180 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-certs\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:47.568455 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.568267 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:52:47.568455 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.568286 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:52:47.568727 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.568704 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-data" (OuterVolumeSpecName: "istio-data") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:52:47.569929 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.569895 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb4197de-6a46-4042-a40d-71df4110d5db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:52:47.570067 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.570049 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:52:47.570156 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.570137 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-kube-api-access-5cn5h" (OuterVolumeSpecName: "kube-api-access-5cn5h") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "kube-api-access-5cn5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:52:47.570363 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.570341 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cb4197de-6a46-4042-a40d-71df4110d5db" (UID: "cb4197de-6a46-4042-a40d-71df4110d5db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:52:47.668964 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.668922 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-data\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:47.668964 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.668954 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-workload-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:47.668964 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.668966 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-istio-envoy\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:47.669245 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.668979 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-istio-token\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:47.669245 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.668991 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb4197de-6a46-4042-a40d-71df4110d5db-istio-podinfo\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:47.669245 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.669001 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb4197de-6a46-4042-a40d-71df4110d5db-credential-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:47.669245 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:47.669013 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cn5h\" (UniqueName: \"kubernetes.io/projected/cb4197de-6a46-4042-a40d-71df4110d5db-kube-api-access-5cn5h\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:48.267254 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.267197 2565 generic.go:356] "Generic (PLEG): container finished" podID="cb4197de-6a46-4042-a40d-71df4110d5db" containerID="1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da" exitCode=0 Jul 06 19:52:48.267428 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.267273 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" Jul 06 19:52:48.267428 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.267281 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" event={"ID":"cb4197de-6a46-4042-a40d-71df4110d5db","Type":"ContainerDied","Data":"1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da"} Jul 06 19:52:48.267428 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.267360 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr" event={"ID":"cb4197de-6a46-4042-a40d-71df4110d5db","Type":"ContainerDied","Data":"0a8ac8462ed8e48e37e307aafecb4879895339fddc7504c7e8ac84562c2d1afe"} Jul 06 19:52:48.267428 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.267382 2565 scope.go:117] "RemoveContainer" containerID="1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da" Jul 06 19:52:48.277055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.277035 2565 scope.go:117] "RemoveContainer" containerID="d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2" Jul 06 19:52:48.285393 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.285373 2565 scope.go:117] "RemoveContainer" containerID="1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da" Jul 06 19:52:48.285702 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:52:48.285675 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da\": container with ID starting with 1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da not found: ID does not exist" containerID="1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da" Jul 06 19:52:48.285749 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.285711 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da"} err="failed to get container status \"1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da\": rpc error: code = NotFound desc = could not find container \"1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da\": container with ID starting with 1262e4cfe829a50cc1361471aa6fdf3cdba5fc958cb107d4a133e8f7bbb2b1da not found: ID does not exist" Jul 06 19:52:48.285749 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.285730 2565 scope.go:117] "RemoveContainer" containerID="d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2" Jul 06 19:52:48.285968 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:52:48.285952 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2\": container with ID starting with d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2 not found: ID does not exist" containerID="d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2" Jul 06 19:52:48.286009 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.285975 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2"} err="failed to get container status \"d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2\": rpc error: code = NotFound desc = could not find container \"d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2\": container with ID starting with d8b04c014334f2101cbae9b407d14aa2277d28f12a278b8e51b86cb23e9c45b2 not found: ID does not exist" Jul 06 19:52:48.295020 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.294984 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr"] Jul 06 19:52:48.305282 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:48.305256 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a583ee7e-openshift-default-59fb65bd5c-t79rr"] Jul 06 19:52:49.208798 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:49.208758 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb4197de-6a46-4042-a40d-71df4110d5db" path="/var/lib/kubelet/pods/cb4197de-6a46-4042-a40d-71df4110d5db/volumes" Jul 06 19:52:50.389684 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.389647 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d"] Jul 06 19:52:50.390066 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.390002 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb4197de-6a46-4042-a40d-71df4110d5db" containerName="istio-proxy" Jul 06 19:52:50.390066 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.390014 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb4197de-6a46-4042-a40d-71df4110d5db" containerName="istio-proxy" Jul 06 19:52:50.390136 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.390104 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb4197de-6a46-4042-a40d-71df4110d5db" containerName="istio-proxy" Jul 06 19:52:50.393027 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.393005 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.396017 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.395992 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-26edcf53-serving-cert\"" Jul 06 19:52:50.406458 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.406423 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d"] Jul 06 19:52:50.495292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.495256 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76d64c3e-dda4-4863-afcf-b699c4c68a97-maas-api-tls\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.495479 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.495378 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnmc4\" (UniqueName: \"kubernetes.io/projected/76d64c3e-dda4-4863-afcf-b699c4c68a97-kube-api-access-tnmc4\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.495479 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.495414 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76d64c3e-dda4-4863-afcf-b699c4c68a97-openshift-service-ca\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.596719 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.596683 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76d64c3e-dda4-4863-afcf-b699c4c68a97-maas-api-tls\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.596903 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.596762 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnmc4\" (UniqueName: \"kubernetes.io/projected/76d64c3e-dda4-4863-afcf-b699c4c68a97-kube-api-access-tnmc4\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.596903 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.596787 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76d64c3e-dda4-4863-afcf-b699c4c68a97-openshift-service-ca\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.597445 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.597421 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76d64c3e-dda4-4863-afcf-b699c4c68a97-openshift-service-ca\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.599168 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.599139 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76d64c3e-dda4-4863-afcf-b699c4c68a97-maas-api-tls\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.609032 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.609004 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnmc4\" (UniqueName: \"kubernetes.io/projected/76d64c3e-dda4-4863-afcf-b699c4c68a97-kube-api-access-tnmc4\") pod \"maas-api-e2e-derive-26edcf53-564989f675-qwn4d\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.703929 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.703840 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:50.840108 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:50.840077 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d"] Jul 06 19:52:50.842518 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:52:50.842485 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d64c3e_dda4_4863_afcf_b699c4c68a97.slice/crio-0351d60275e4c810a0a568f6f3143c02b568f554749c102fac808dccc7f4437b WatchSource:0}: Error finding container 0351d60275e4c810a0a568f6f3143c02b568f554749c102fac808dccc7f4437b: Status 404 returned error can't find the container with id 0351d60275e4c810a0a568f6f3143c02b568f554749c102fac808dccc7f4437b Jul 06 19:52:51.283562 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:51.283527 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" event={"ID":"76d64c3e-dda4-4863-afcf-b699c4c68a97","Type":"ContainerStarted","Data":"f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413"} Jul 06 19:52:51.283750 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:51.283586 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" event={"ID":"76d64c3e-dda4-4863-afcf-b699c4c68a97","Type":"ContainerStarted","Data":"0351d60275e4c810a0a568f6f3143c02b568f554749c102fac808dccc7f4437b"} Jul 06 19:52:51.283750 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:51.283637 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:51.304703 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:51.304648 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" podStartSLOduration=0.964482554 podStartE2EDuration="1.304630047s" totalImagesPullingTime="340.147493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:52:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:52:51.303801654 +0000 UTC m=+1602.678257595" watchObservedRunningTime="2026-07-06 19:52:51.304630047 +0000 UTC m=+1602.679086038" Jul 06 19:52:55.064834 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.064798 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d"] Jul 06 19:52:55.065385 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.065067 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" podUID="76d64c3e-dda4-4863-afcf-b699c4c68a97" containerName="maas-api" containerID="cri-o://f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413" gracePeriod=30 Jul 06 19:52:55.301564 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.301541 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:55.304178 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.304150 2565 generic.go:356] "Generic (PLEG): container finished" podID="76d64c3e-dda4-4863-afcf-b699c4c68a97" containerID="f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413" exitCode=0 Jul 06 19:52:55.304322 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.304189 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" Jul 06 19:52:55.304322 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.304243 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" event={"ID":"76d64c3e-dda4-4863-afcf-b699c4c68a97","Type":"ContainerDied","Data":"f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413"} Jul 06 19:52:55.304322 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.304302 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d" event={"ID":"76d64c3e-dda4-4863-afcf-b699c4c68a97","Type":"ContainerDied","Data":"0351d60275e4c810a0a568f6f3143c02b568f554749c102fac808dccc7f4437b"} Jul 06 19:52:55.304436 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.304326 2565 scope.go:117] "RemoveContainer" containerID="f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413" Jul 06 19:52:55.312202 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.312187 2565 scope.go:117] "RemoveContainer" containerID="f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413" Jul 06 19:52:55.312471 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:52:55.312446 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413\": container with ID starting with f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413 not found: ID does not exist" containerID="f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413" Jul 06 19:52:55.312521 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.312472 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413"} err="failed to get container status \"f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413\": rpc error: code = NotFound desc = could not find container \"f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413\": container with ID starting with f8e86e08478d47c069d62f5fc5cc51d64e3f20a65b7df8a3fcd16719ea05a413 not found: ID does not exist" Jul 06 19:52:55.442462 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.442428 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnmc4\" (UniqueName: \"kubernetes.io/projected/76d64c3e-dda4-4863-afcf-b699c4c68a97-kube-api-access-tnmc4\") pod \"76d64c3e-dda4-4863-afcf-b699c4c68a97\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " Jul 06 19:52:55.442655 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.442493 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76d64c3e-dda4-4863-afcf-b699c4c68a97-maas-api-tls\") pod \"76d64c3e-dda4-4863-afcf-b699c4c68a97\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " Jul 06 19:52:55.442655 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.442545 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76d64c3e-dda4-4863-afcf-b699c4c68a97-openshift-service-ca\") pod \"76d64c3e-dda4-4863-afcf-b699c4c68a97\" (UID: \"76d64c3e-dda4-4863-afcf-b699c4c68a97\") " Jul 06 19:52:55.442951 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.442915 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d64c3e-dda4-4863-afcf-b699c4c68a97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "76d64c3e-dda4-4863-afcf-b699c4c68a97" (UID: "76d64c3e-dda4-4863-afcf-b699c4c68a97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:52:55.444570 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.444549 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d64c3e-dda4-4863-afcf-b699c4c68a97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "76d64c3e-dda4-4863-afcf-b699c4c68a97" (UID: "76d64c3e-dda4-4863-afcf-b699c4c68a97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:52:55.444662 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.444565 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d64c3e-dda4-4863-afcf-b699c4c68a97-kube-api-access-tnmc4" (OuterVolumeSpecName: "kube-api-access-tnmc4") pod "76d64c3e-dda4-4863-afcf-b699c4c68a97" (UID: "76d64c3e-dda4-4863-afcf-b699c4c68a97"). InnerVolumeSpecName "kube-api-access-tnmc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:52:55.543599 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.543556 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76d64c3e-dda4-4863-afcf-b699c4c68a97-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:55.543599 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.543591 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnmc4\" (UniqueName: \"kubernetes.io/projected/76d64c3e-dda4-4863-afcf-b699c4c68a97-kube-api-access-tnmc4\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:55.543599 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.543602 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76d64c3e-dda4-4863-afcf-b699c4c68a97-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:52:55.628360 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.628327 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d"] Jul 06 19:52:55.633034 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:55.633008 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-26edcf53-564989f675-qwn4d"] Jul 06 19:52:57.203578 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:52:57.203547 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d64c3e-dda4-4863-afcf-b699c4c68a97" path="/var/lib/kubelet/pods/76d64c3e-dda4-4863-afcf-b699c4c68a97/volumes" Jul 06 19:56:08.642277 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.642239 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q"] Jul 06 19:56:08.642805 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.642785 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb4197de-6a46-4042-a40d-71df4110d5db" containerName="istio-proxy" Jul 06 19:56:08.642805 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.642805 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb4197de-6a46-4042-a40d-71df4110d5db" containerName="istio-proxy" Jul 06 19:56:08.642925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.642861 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d64c3e-dda4-4863-afcf-b699c4c68a97" containerName="maas-api" Jul 06 19:56:08.642925 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.642870 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d64c3e-dda4-4863-afcf-b699c4c68a97" containerName="maas-api" Jul 06 19:56:08.643036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.642962 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d64c3e-dda4-4863-afcf-b699c4c68a97" containerName="maas-api" Jul 06 19:56:08.643036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.642980 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb4197de-6a46-4042-a40d-71df4110d5db" containerName="istio-proxy" Jul 06 19:56:08.646491 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.646467 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.649433 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.649412 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-d56cc522-serving-cert\"" Jul 06 19:56:08.656282 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.656256 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q"] Jul 06 19:56:08.784083 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.784047 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-openshift-service-ca\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.784083 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.784082 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9knq\" (UniqueName: \"kubernetes.io/projected/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-kube-api-access-z9knq\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.784354 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.784123 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-maas-api-tls\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.885532 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.885497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-maas-api-tls\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.885760 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.885605 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-openshift-service-ca\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.885760 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.885634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9knq\" (UniqueName: \"kubernetes.io/projected/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-kube-api-access-z9knq\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.886318 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.886287 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-openshift-service-ca\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.888002 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.887978 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-maas-api-tls\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.901036 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.900975 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9knq\" (UniqueName: \"kubernetes.io/projected/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-kube-api-access-z9knq\") pod \"maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:08.958071 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:08.958033 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:09.095869 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:09.095841 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q"] Jul 06 19:56:09.098408 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:56:09.098368 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b81ac95_43ea_473e_9f17_4e5c4ce62e0c.slice/crio-cc0c24ea5952154315d93ded6517564abf414aaf0d29f3cfa5d209619d96b899 WatchSource:0}: Error finding container cc0c24ea5952154315d93ded6517564abf414aaf0d29f3cfa5d209619d96b899: Status 404 returned error can't find the container with id cc0c24ea5952154315d93ded6517564abf414aaf0d29f3cfa5d209619d96b899 Jul 06 19:56:09.213601 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:09.213509 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:56:09.214091 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:09.214067 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 19:56:09.216009 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:09.215977 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" event={"ID":"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c","Type":"ContainerStarted","Data":"cc0c24ea5952154315d93ded6517564abf414aaf0d29f3cfa5d209619d96b899"} Jul 06 19:56:09.219559 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:09.219541 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:56:09.219833 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:09.219816 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 19:56:10.222753 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:10.222711 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" event={"ID":"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c","Type":"ContainerStarted","Data":"541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3"} Jul 06 19:56:10.223160 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:10.222797 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:10.247043 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:10.246989 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" podStartSLOduration=1.862211456 podStartE2EDuration="2.246972599s" totalImagesPullingTime="384.761143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:10.245003337 +0000 UTC m=+1801.619459301" watchObservedRunningTime="2026-07-06 19:56:10.246972599 +0000 UTC m=+1801.621428538" Jul 06 19:56:16.232496 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:16.232462 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:21.360234 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.360185 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q"] Jul 06 19:56:21.360686 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.360468 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" podUID="7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" containerName="maas-api" containerID="cri-o://541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3" gracePeriod=30 Jul 06 19:56:21.602027 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.601995 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:21.709002 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.708918 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-openshift-service-ca\") pod \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " Jul 06 19:56:21.709002 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.708948 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-maas-api-tls\") pod \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " Jul 06 19:56:21.709261 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.709055 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9knq\" (UniqueName: \"kubernetes.io/projected/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-kube-api-access-z9knq\") pod \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\" (UID: \"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c\") " Jul 06 19:56:21.709331 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.709312 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" (UID: "7b81ac95-43ea-473e-9f17-4e5c4ce62e0c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:56:21.711133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.711113 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" (UID: "7b81ac95-43ea-473e-9f17-4e5c4ce62e0c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:56:21.711261 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.711158 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-kube-api-access-z9knq" (OuterVolumeSpecName: "kube-api-access-z9knq") pod "7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" (UID: "7b81ac95-43ea-473e-9f17-4e5c4ce62e0c"). InnerVolumeSpecName "kube-api-access-z9knq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:56:21.809743 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.809706 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:56:21.809743 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.809740 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:56:21.809743 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:21.809752 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9knq\" (UniqueName: \"kubernetes.io/projected/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c-kube-api-access-z9knq\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:56:22.280292 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.280256 2565 generic.go:356] "Generic (PLEG): container finished" podID="7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" containerID="541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3" exitCode=0 Jul 06 19:56:22.280471 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.280305 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" Jul 06 19:56:22.280471 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.280339 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" event={"ID":"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c","Type":"ContainerDied","Data":"541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3"} Jul 06 19:56:22.280471 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.280391 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q" event={"ID":"7b81ac95-43ea-473e-9f17-4e5c4ce62e0c","Type":"ContainerDied","Data":"cc0c24ea5952154315d93ded6517564abf414aaf0d29f3cfa5d209619d96b899"} Jul 06 19:56:22.280471 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.280413 2565 scope.go:117] "RemoveContainer" containerID="541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3" Jul 06 19:56:22.289870 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.289849 2565 scope.go:117] "RemoveContainer" containerID="541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3" Jul 06 19:56:22.290146 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:56:22.290127 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3\": container with ID starting with 541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3 not found: ID does not exist" containerID="541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3" Jul 06 19:56:22.290193 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.290155 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3"} err="failed to get container status \"541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3\": rpc error: code = NotFound desc = could not find container \"541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3\": container with ID starting with 541822b44a55d1601ad1be8db53cf8c2b95401a36393e72edaf51e7eb648bbe3 not found: ID does not exist" Jul 06 19:56:22.311055 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.311023 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q"] Jul 06 19:56:22.320553 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:22.320527 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-d56cc522-5fbcc45789-p4t7q"] Jul 06 19:56:23.204485 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:23.204452 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" path="/var/lib/kubelet/pods/7b81ac95-43ea-473e-9f17-4e5c4ce62e0c/volumes" Jul 06 19:56:48.108688 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.108651 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct"] Jul 06 19:56:48.109149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.108981 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" containerName="maas-api" Jul 06 19:56:48.109149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.108990 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" containerName="maas-api" Jul 06 19:56:48.109149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.109067 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b81ac95-43ea-473e-9f17-4e5c4ce62e0c" containerName="maas-api" Jul 06 19:56:48.112079 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.112058 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.114973 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.114940 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9c52a789-openshift-default-dockercfg-nl467\"" Jul 06 19:56:48.125557 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.125532 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct"] Jul 06 19:56:48.252970 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.252924 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-certs\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.253149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.252997 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-envoy\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.253149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.253026 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-credential-socket\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.253149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.253068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-data\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.253149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.253086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-podinfo\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.253149 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.253137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-token\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.253346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.253161 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf8lr\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-kube-api-access-jf8lr\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.253346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.253191 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istiod-ca-cert\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.253346 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.253244 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-socket\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.353713 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353686 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-envoy\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.353882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353721 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-credential-socket\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.353882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-data\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.353882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353792 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-podinfo\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.353882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353835 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-token\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.353882 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353863 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf8lr\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-kube-api-access-jf8lr\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.354122 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353889 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istiod-ca-cert\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.354122 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353914 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-socket\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.354122 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.353976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-certs\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.354299 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.354236 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-credential-socket\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.354299 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.354280 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-data\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.354402 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.354327 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-certs\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.354556 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.354533 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-socket\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.354732 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.354714 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istiod-ca-cert\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.355858 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.355840 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-envoy\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.356162 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.356144 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-podinfo\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.363592 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.363505 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-token\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.363750 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.363615 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf8lr\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-kube-api-access-jf8lr\") pod \"e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.422845 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.422808 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:48.573820 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.573795 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct"] Jul 06 19:56:48.576346 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:56:48.576316 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb56c0ad7_6b1d_4cb2_af61_8b3d8e1dfbb8.slice/crio-669ae64e8649c549d9b35ccf9ce0d761208b52a8fc20195dd7a71a02aebb9354 WatchSource:0}: Error finding container 669ae64e8649c549d9b35ccf9ce0d761208b52a8fc20195dd7a71a02aebb9354: Status 404 returned error can't find the container with id 669ae64e8649c549d9b35ccf9ce0d761208b52a8fc20195dd7a71a02aebb9354 Jul 06 19:56:48.578878 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.578846 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:56:48.578966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.578910 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:56:48.578966 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:48.578940 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:56:49.405552 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:49.405513 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" event={"ID":"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8","Type":"ContainerStarted","Data":"af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c"} Jul 06 19:56:49.405964 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:49.405573 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" event={"ID":"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8","Type":"ContainerStarted","Data":"669ae64e8649c549d9b35ccf9ce0d761208b52a8fc20195dd7a71a02aebb9354"} Jul 06 19:56:49.423788 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:49.423756 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:49.429497 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:49.429074 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:49.430141 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:49.430098 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" podStartSLOduration=1.4300824140000001 podStartE2EDuration="1.430082414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:49.429005334 +0000 UTC m=+1840.803461274" watchObservedRunningTime="2026-07-06 19:56:49.430082414 +0000 UTC m=+1840.804538407" Jul 06 19:56:50.410175 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:50.410135 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:56:50.411117 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:56:50.411101 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:57:05.411616 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:05.411530 2565 patch_prober.go:28] interesting pod/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:57:05.411616 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:05.411595 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 19:57:05.483181 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:05.483146 2565 generic.go:356] "Generic (PLEG): container finished" podID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerID="af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c" exitCode=0 Jul 06 19:57:05.483394 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:05.483234 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" event={"ID":"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8","Type":"ContainerDied","Data":"af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c"} Jul 06 19:57:05.483667 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:05.483654 2565 scope.go:117] "RemoveContainer" containerID="af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c" Jul 06 19:57:05.484682 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:05.484656 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:57:05.484764 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:05.484720 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:57:05.484806 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:05.484760 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:57:06.491228 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:06.491183 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" event={"ID":"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8","Type":"ContainerStarted","Data":"331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2"} Jul 06 19:57:07.423279 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:07.423238 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:57:07.423496 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:07.423379 2565 patch_prober.go:28] interesting pod/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:57:07.423496 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:07.423428 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 19:57:08.424252 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:08.423693 2565 patch_prober.go:28] interesting pod/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:57:08.424252 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:08.423776 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 19:57:09.423626 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:09.423573 2565 patch_prober.go:28] interesting pod/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:57:09.423850 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:09.423649 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 19:57:10.423685 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:10.423636 2565 patch_prober.go:28] interesting pod/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:57:10.424184 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:10.423722 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 19:57:11.427971 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:11.427938 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:57:11.428361 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:11.428328 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:57:11.429090 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:57:11.429071 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:58:20.053109 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:20.053071 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct"] Jul 06 19:58:20.053618 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:20.053326 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" containerID="cri-o://331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2" gracePeriod=30 Jul 06 19:58:25.317075 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.317055 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:58:25.451936 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.451895 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-credential-socket\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.452125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.451949 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-data\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.452125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.451985 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-socket\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.452125 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.452017 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-podinfo\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.452343 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.452278 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:58:25.452343 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.452329 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:58:25.452600 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.452070 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istiod-ca-cert\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.452669 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.452638 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:58:25.452669 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.452656 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf8lr\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-kube-api-access-jf8lr\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.452767 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.452713 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-envoy\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.453039 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453007 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-data" (OuterVolumeSpecName: "istio-data") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:58:25.453161 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453043 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-token\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.453161 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453118 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-certs\") pod \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\" (UID: \"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8\") " Jul 06 19:58:25.453396 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453374 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:58:25.453519 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453501 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-certs\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.453590 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453525 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-credential-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.453590 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453541 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-data\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.453590 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453556 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-workload-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.453590 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.453569 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istiod-ca-cert\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.454950 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.454906 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:58:25.455599 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.455569 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:58:25.455837 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.455804 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-kube-api-access-jf8lr" (OuterVolumeSpecName: "kube-api-access-jf8lr") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "kube-api-access-jf8lr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:58:25.456370 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.456339 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-token" (OuterVolumeSpecName: "istio-token") pod "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" (UID: "b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:58:25.554181 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.554139 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-podinfo\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.554387 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.554196 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf8lr\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-kube-api-access-jf8lr\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.554387 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.554234 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-envoy\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.554387 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.554243 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8-istio-token\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 19:58:25.865316 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.865278 2565 generic.go:356] "Generic (PLEG): container finished" podID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerID="331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2" exitCode=0 Jul 06 19:58:25.865491 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.865334 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" Jul 06 19:58:25.865491 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.865371 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" event={"ID":"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8","Type":"ContainerDied","Data":"331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2"} Jul 06 19:58:25.865491 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.865429 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct" event={"ID":"b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8","Type":"ContainerDied","Data":"669ae64e8649c549d9b35ccf9ce0d761208b52a8fc20195dd7a71a02aebb9354"} Jul 06 19:58:25.865491 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.865455 2565 scope.go:117] "RemoveContainer" containerID="331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2" Jul 06 19:58:25.874629 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.874612 2565 scope.go:117] "RemoveContainer" containerID="af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c" Jul 06 19:58:25.882671 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.882651 2565 scope.go:117] "RemoveContainer" containerID="331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2" Jul 06 19:58:25.882948 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:58:25.882930 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2\": container with ID starting with 331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2 not found: ID does not exist" containerID="331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2" Jul 06 19:58:25.882986 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.882955 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2"} err="failed to get container status \"331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2\": rpc error: code = NotFound desc = could not find container \"331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2\": container with ID starting with 331ab2c0c0b35dd6259a0d1d6f284742fd710cb505af4644af321674cd049ce2 not found: ID does not exist" Jul 06 19:58:25.882986 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.882972 2565 scope.go:117] "RemoveContainer" containerID="af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c" Jul 06 19:58:25.883202 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:58:25.883183 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c\": container with ID starting with af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c not found: ID does not exist" containerID="af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c" Jul 06 19:58:25.883268 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.883231 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c"} err="failed to get container status \"af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c\": rpc error: code = NotFound desc = could not find container \"af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c\": container with ID starting with af012e90e6e6e63b49fe958ffdba3b770659703e7bbbaf425e3b1afba9c6541c not found: ID does not exist" Jul 06 19:58:25.890482 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.890457 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct"] Jul 06 19:58:25.900450 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:25.900426 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9c52a789-openshift-default-b67ff8df5-ps9ct"] Jul 06 19:58:27.205067 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:58:27.205035 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" path="/var/lib/kubelet/pods/b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8/volumes" Jul 06 19:59:16.300853 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.300820 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2"] Jul 06 19:59:16.301312 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.301154 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" Jul 06 19:59:16.301312 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.301165 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" Jul 06 19:59:16.301312 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.301178 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" Jul 06 19:59:16.301312 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.301184 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" Jul 06 19:59:16.301312 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.301255 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" Jul 06 19:59:16.301312 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.301264 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b56c0ad7-6b1d-4cb2-af61-8b3d8e1dfbb8" containerName="istio-proxy" Jul 06 19:59:16.304126 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.304109 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.307166 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.307145 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d188f8-openshift-default-dockercfg-vbbkr\"" Jul 06 19:59:16.321174 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.321145 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2"] Jul 06 19:59:16.426877 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.426843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istiod-ca-cert\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.426877 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.426886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-socket\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.427133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.426918 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-token\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.427133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.426991 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cksgn\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-kube-api-access-cksgn\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.427133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.427086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-podinfo\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.427133 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.427125 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-credential-socket\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.427421 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.427149 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-certs\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.427421 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.427182 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-envoy\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.427421 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.427381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-data\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528645 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-data\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528645 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528655 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istiod-ca-cert\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528680 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-socket\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528709 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-token\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528736 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cksgn\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-kube-api-access-cksgn\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-podinfo\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-credential-socket\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528860 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-certs\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.528892 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.528884 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-envoy\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.529189 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.529042 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-data\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.529189 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.529083 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-socket\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.529340 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.529306 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-credential-socket\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.529423 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.529403 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-certs\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.529483 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.529454 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istiod-ca-cert\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.531341 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.531316 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-envoy\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.531495 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.531478 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-podinfo\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.540654 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.540622 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-token\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.540782 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.540697 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cksgn\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-kube-api-access-cksgn\") pod \"e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.614286 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.614247 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:16.764438 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.764409 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2"] Jul 06 19:59:16.766508 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:59:16.766476 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee3a4bd_68b2_4bd7_a5be_b6f3f21293ff.slice/crio-ad2cad2db8ba19bc547db671714962105315c549c5370da76397c9f44080eed6 WatchSource:0}: Error finding container ad2cad2db8ba19bc547db671714962105315c549c5370da76397c9f44080eed6: Status 404 returned error can't find the container with id ad2cad2db8ba19bc547db671714962105315c549c5370da76397c9f44080eed6 Jul 06 19:59:16.768730 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.768697 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:59:16.768821 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.768768 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:59:16.768821 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:16.768799 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:59:17.103446 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:17.103402 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" event={"ID":"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff","Type":"ContainerStarted","Data":"a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932"} Jul 06 19:59:17.103631 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:17.103471 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" event={"ID":"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff","Type":"ContainerStarted","Data":"ad2cad2db8ba19bc547db671714962105315c549c5370da76397c9f44080eed6"} Jul 06 19:59:17.126800 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:17.126735 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" podStartSLOduration=1.126714744 podStartE2EDuration="1.126714744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:17.125249215 +0000 UTC m=+1988.499705179" watchObservedRunningTime="2026-07-06 19:59:17.126714744 +0000 UTC m=+1988.501170683" Jul 06 19:59:17.614883 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:17.614852 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:17.620259 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:17.620231 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:18.107864 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:18.107824 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:18.108955 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:18.108933 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:19.248759 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.248708 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j"] Jul 06 19:59:19.252848 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.252817 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.255853 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.255823 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-d188f8-serving-cert\"" Jul 06 19:59:19.261402 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.261370 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j"] Jul 06 19:59:19.350172 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.350135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4de06a8-972b-45fb-b852-4f46ed56a22a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.350172 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.350178 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tk4l\" (UniqueName: \"kubernetes.io/projected/d4de06a8-972b-45fb-b852-4f46ed56a22a-kube-api-access-7tk4l\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.350437 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.350246 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4de06a8-972b-45fb-b852-4f46ed56a22a-maas-api-tls\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.450773 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.450731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tk4l\" (UniqueName: \"kubernetes.io/projected/d4de06a8-972b-45fb-b852-4f46ed56a22a-kube-api-access-7tk4l\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.450952 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.450783 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4de06a8-972b-45fb-b852-4f46ed56a22a-maas-api-tls\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.450952 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.450858 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4de06a8-972b-45fb-b852-4f46ed56a22a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.451510 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.451492 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4de06a8-972b-45fb-b852-4f46ed56a22a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.453369 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.453351 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4de06a8-972b-45fb-b852-4f46ed56a22a-maas-api-tls\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.465188 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.465161 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tk4l\" (UniqueName: \"kubernetes.io/projected/d4de06a8-972b-45fb-b852-4f46ed56a22a-kube-api-access-7tk4l\") pod \"maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.566256 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.566197 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:19.713794 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.713770 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j"] Jul 06 19:59:19.716276 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:59:19.716246 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4de06a8_972b_45fb_b852_4f46ed56a22a.slice/crio-93fbef9de8f0aa567423c46cc0f759fac26189d8c73456ab6774eb0c48a84e34 WatchSource:0}: Error finding container 93fbef9de8f0aa567423c46cc0f759fac26189d8c73456ab6774eb0c48a84e34: Status 404 returned error can't find the container with id 93fbef9de8f0aa567423c46cc0f759fac26189d8c73456ab6774eb0c48a84e34 Jul 06 19:59:19.717791 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:19.717774 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:59:20.118517 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:20.118470 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" event={"ID":"d4de06a8-972b-45fb-b852-4f46ed56a22a","Type":"ContainerStarted","Data":"93fbef9de8f0aa567423c46cc0f759fac26189d8c73456ab6774eb0c48a84e34"} Jul 06 19:59:21.124958 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:21.124925 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" event={"ID":"d4de06a8-972b-45fb-b852-4f46ed56a22a","Type":"ContainerStarted","Data":"6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2"} Jul 06 19:59:21.125432 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:21.125015 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:21.145203 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:21.145147 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" podStartSLOduration=1.6634767049999999 podStartE2EDuration="2.1451283s" totalImagesPullingTime="481.651595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:21.144148094 +0000 UTC m=+1992.518604054" watchObservedRunningTime="2026-07-06 19:59:21.1451283 +0000 UTC m=+1992.519584266" Jul 06 19:59:27.135857 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.135823 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 19:59:27.565283 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.565255 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn"] Jul 06 19:59:27.568978 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.568949 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:27.572031 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.572002 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-62de5f-serving-cert\"" Jul 06 19:59:27.583146 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.583114 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn"] Jul 06 19:59:27.619185 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.619144 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:27.619527 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.619493 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a194f34-a47a-4c27-8590-0b4ae931b922-openshift-service-ca\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:27.619697 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.619680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nnjn\" (UniqueName: \"kubernetes.io/projected/9a194f34-a47a-4c27-8590-0b4ae931b922-kube-api-access-6nnjn\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:27.720488 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.720443 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:27.720795 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.720771 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a194f34-a47a-4c27-8590-0b4ae931b922-openshift-service-ca\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:27.720939 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.720924 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nnjn\" (UniqueName: \"kubernetes.io/projected/9a194f34-a47a-4c27-8590-0b4ae931b922-kube-api-access-6nnjn\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:27.721450 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:59:27.720674 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-62de5f-serving-cert: secret "maas-api-e2e-inf-b-62de5f-serving-cert" not found Jul 06 19:59:27.721694 ip-10-0-143-100 kubenswrapper[2565]: E0706 19:59:27.721679 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls podName:9a194f34-a47a-4c27-8590-0b4ae931b922 nodeName:}" failed. No retries permitted until 2026-07-06 19:59:28.221649177 +0000 UTC m=+1999.596105108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls") pod "maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" (UID: "9a194f34-a47a-4c27-8590-0b4ae931b922") : secret "maas-api-e2e-inf-b-62de5f-serving-cert" not found Jul 06 19:59:27.722633 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.722597 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a194f34-a47a-4c27-8590-0b4ae931b922-openshift-service-ca\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:27.735465 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:27.735433 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nnjn\" (UniqueName: \"kubernetes.io/projected/9a194f34-a47a-4c27-8590-0b4ae931b922-kube-api-access-6nnjn\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:28.226920 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:28.226875 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:28.236267 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:28.231057 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls\") pod \"maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:28.484388 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:28.484292 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:28.678915 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:28.678851 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn"] Jul 06 19:59:28.681390 ip-10-0-143-100 kubenswrapper[2565]: W0706 19:59:28.681361 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a194f34_a47a_4c27_8590_0b4ae931b922.slice/crio-bfc8f920820f07b1538abbc61a3fb57fd5b9fb223bde7a296d4cc73764355eca WatchSource:0}: Error finding container bfc8f920820f07b1538abbc61a3fb57fd5b9fb223bde7a296d4cc73764355eca: Status 404 returned error can't find the container with id bfc8f920820f07b1538abbc61a3fb57fd5b9fb223bde7a296d4cc73764355eca Jul 06 19:59:29.164543 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:29.164499 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" event={"ID":"9a194f34-a47a-4c27-8590-0b4ae931b922","Type":"ContainerStarted","Data":"bfc8f920820f07b1538abbc61a3fb57fd5b9fb223bde7a296d4cc73764355eca"} Jul 06 19:59:30.171321 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:30.171285 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" event={"ID":"9a194f34-a47a-4c27-8590-0b4ae931b922","Type":"ContainerStarted","Data":"c5a5ddc37d344ed92b3641c5470364be06bdc0b4d7238ccc814686a337bb88fa"} Jul 06 19:59:30.171819 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:30.171451 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:30.192111 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:30.192061 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" podStartSLOduration=2.65783426 podStartE2EDuration="3.192044168s" totalImagesPullingTime="534.209908ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:30.189731653 +0000 UTC m=+2001.564187596" watchObservedRunningTime="2026-07-06 19:59:30.192044168 +0000 UTC m=+2001.566500108" Jul 06 19:59:34.109838 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:34.109794 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 19:59:34.110264 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:34.109863 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:59:35.198136 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:35.198101 2565 generic.go:356] "Generic (PLEG): container finished" podID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerID="a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932" exitCode=0 Jul 06 19:59:35.203517 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:35.203484 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" event={"ID":"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff","Type":"ContainerDied","Data":"a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932"} Jul 06 19:59:35.203904 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:35.203887 2565 scope.go:117] "RemoveContainer" containerID="a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932" Jul 06 19:59:35.204975 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:35.204946 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:59:35.205052 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:35.205033 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:59:35.205113 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:35.205098 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:59:36.183792 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:36.183756 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 19:59:36.205484 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:36.205441 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" event={"ID":"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff","Type":"ContainerStarted","Data":"c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9"} Jul 06 19:59:36.615418 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:36.615375 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:36.615641 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:36.615516 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 06 19:59:36.615641 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:36.615565 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 06 19:59:37.617238 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:37.615907 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 06 19:59:37.617238 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:37.615971 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 06 19:59:38.615458 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:38.615415 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 06 19:59:38.615682 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:38.615488 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 06 19:59:39.615488 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:39.615445 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 06 19:59:39.616023 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:39.615518 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 06 19:59:40.618266 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:40.618233 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:40.618652 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:40.618500 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 19:59:40.619264 ip-10-0-143-100 kubenswrapper[2565]: I0706 19:59:40.619248 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 20:00:22.621587 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.621551 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j"] Jul 06 20:00:22.622483 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.622152 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" podUID="d4de06a8-972b-45fb-b852-4f46ed56a22a" containerName="maas-api" containerID="cri-o://6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2" gracePeriod=30 Jul 06 20:00:22.880207 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.880144 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 20:00:22.962447 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.962413 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4de06a8-972b-45fb-b852-4f46ed56a22a-maas-api-tls\") pod \"d4de06a8-972b-45fb-b852-4f46ed56a22a\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " Jul 06 20:00:22.962616 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.962459 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4de06a8-972b-45fb-b852-4f46ed56a22a-openshift-service-ca\") pod \"d4de06a8-972b-45fb-b852-4f46ed56a22a\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " Jul 06 20:00:22.962616 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.962513 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tk4l\" (UniqueName: \"kubernetes.io/projected/d4de06a8-972b-45fb-b852-4f46ed56a22a-kube-api-access-7tk4l\") pod \"d4de06a8-972b-45fb-b852-4f46ed56a22a\" (UID: \"d4de06a8-972b-45fb-b852-4f46ed56a22a\") " Jul 06 20:00:22.962856 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.962829 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4de06a8-972b-45fb-b852-4f46ed56a22a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d4de06a8-972b-45fb-b852-4f46ed56a22a" (UID: "d4de06a8-972b-45fb-b852-4f46ed56a22a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:00:22.964483 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.964459 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4de06a8-972b-45fb-b852-4f46ed56a22a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d4de06a8-972b-45fb-b852-4f46ed56a22a" (UID: "d4de06a8-972b-45fb-b852-4f46ed56a22a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:00:22.964779 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:22.964751 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4de06a8-972b-45fb-b852-4f46ed56a22a-kube-api-access-7tk4l" (OuterVolumeSpecName: "kube-api-access-7tk4l") pod "d4de06a8-972b-45fb-b852-4f46ed56a22a" (UID: "d4de06a8-972b-45fb-b852-4f46ed56a22a"). InnerVolumeSpecName "kube-api-access-7tk4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:00:23.063904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.063868 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4de06a8-972b-45fb-b852-4f46ed56a22a-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:00:23.063904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.063899 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4de06a8-972b-45fb-b852-4f46ed56a22a-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:00:23.063904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.063909 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tk4l\" (UniqueName: \"kubernetes.io/projected/d4de06a8-972b-45fb-b852-4f46ed56a22a-kube-api-access-7tk4l\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:00:23.429685 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.429648 2565 generic.go:356] "Generic (PLEG): container finished" podID="d4de06a8-972b-45fb-b852-4f46ed56a22a" containerID="6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2" exitCode=0 Jul 06 20:00:23.429864 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.429701 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" Jul 06 20:00:23.429864 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.429735 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" event={"ID":"d4de06a8-972b-45fb-b852-4f46ed56a22a","Type":"ContainerDied","Data":"6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2"} Jul 06 20:00:23.429864 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.429795 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j" event={"ID":"d4de06a8-972b-45fb-b852-4f46ed56a22a","Type":"ContainerDied","Data":"93fbef9de8f0aa567423c46cc0f759fac26189d8c73456ab6774eb0c48a84e34"} Jul 06 20:00:23.429864 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.429826 2565 scope.go:117] "RemoveContainer" containerID="6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2" Jul 06 20:00:23.438317 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.438288 2565 scope.go:117] "RemoveContainer" containerID="6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2" Jul 06 20:00:23.438600 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:00:23.438580 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2\": container with ID starting with 6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2 not found: ID does not exist" containerID="6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2" Jul 06 20:00:23.438654 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.438609 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2"} err="failed to get container status \"6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2\": rpc error: code = NotFound desc = could not find container \"6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2\": container with ID starting with 6fc8bfd8c634bc9d25c55238830d51e56f895bd54a44ad33ce79308c59731cb2 not found: ID does not exist" Jul 06 20:00:23.449110 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.449079 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j"] Jul 06 20:00:23.451787 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:23.451764 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-d188f8-6fb958dc6d-6dn8j"] Jul 06 20:00:25.207332 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:00:25.207286 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4de06a8-972b-45fb-b852-4f46ed56a22a" path="/var/lib/kubelet/pods/d4de06a8-972b-45fb-b852-4f46ed56a22a/volumes" Jul 06 20:01:05.094402 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:05.094367 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2"] Jul 06 20:01:05.094989 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:05.094600 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" containerID="cri-o://c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9" gracePeriod=30 Jul 06 20:01:06.396880 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.396840 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn"] Jul 06 20:01:06.397295 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.397108 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" podUID="9a194f34-a47a-4c27-8590-0b4ae931b922" containerName="maas-api" containerID="cri-o://c5a5ddc37d344ed92b3641c5470364be06bdc0b4d7238ccc814686a337bb88fa" gracePeriod=30 Jul 06 20:01:06.633135 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.633102 2565 generic.go:356] "Generic (PLEG): container finished" podID="9a194f34-a47a-4c27-8590-0b4ae931b922" containerID="c5a5ddc37d344ed92b3641c5470364be06bdc0b4d7238ccc814686a337bb88fa" exitCode=0 Jul 06 20:01:06.633285 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.633176 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" event={"ID":"9a194f34-a47a-4c27-8590-0b4ae931b922","Type":"ContainerDied","Data":"c5a5ddc37d344ed92b3641c5470364be06bdc0b4d7238ccc814686a337bb88fa"} Jul 06 20:01:06.650240 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.650170 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 20:01:06.783842 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.783791 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nnjn\" (UniqueName: \"kubernetes.io/projected/9a194f34-a47a-4c27-8590-0b4ae931b922-kube-api-access-6nnjn\") pod \"9a194f34-a47a-4c27-8590-0b4ae931b922\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " Jul 06 20:01:06.784051 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.783867 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a194f34-a47a-4c27-8590-0b4ae931b922-openshift-service-ca\") pod \"9a194f34-a47a-4c27-8590-0b4ae931b922\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " Jul 06 20:01:06.784051 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.783886 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls\") pod \"9a194f34-a47a-4c27-8590-0b4ae931b922\" (UID: \"9a194f34-a47a-4c27-8590-0b4ae931b922\") " Jul 06 20:01:06.784271 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.784197 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a194f34-a47a-4c27-8590-0b4ae931b922-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9a194f34-a47a-4c27-8590-0b4ae931b922" (UID: "9a194f34-a47a-4c27-8590-0b4ae931b922"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:01:06.786085 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.786041 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9a194f34-a47a-4c27-8590-0b4ae931b922" (UID: "9a194f34-a47a-4c27-8590-0b4ae931b922"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:01:06.786085 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.786074 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a194f34-a47a-4c27-8590-0b4ae931b922-kube-api-access-6nnjn" (OuterVolumeSpecName: "kube-api-access-6nnjn") pod "9a194f34-a47a-4c27-8590-0b4ae931b922" (UID: "9a194f34-a47a-4c27-8590-0b4ae931b922"). InnerVolumeSpecName "kube-api-access-6nnjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:01:06.885492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.885451 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a194f34-a47a-4c27-8590-0b4ae931b922-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:06.885492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.885482 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a194f34-a47a-4c27-8590-0b4ae931b922-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:06.885492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:06.885492 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nnjn\" (UniqueName: \"kubernetes.io/projected/9a194f34-a47a-4c27-8590-0b4ae931b922-kube-api-access-6nnjn\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:07.638936 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:07.638905 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" Jul 06 20:01:07.639360 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:07.638937 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn" event={"ID":"9a194f34-a47a-4c27-8590-0b4ae931b922","Type":"ContainerDied","Data":"bfc8f920820f07b1538abbc61a3fb57fd5b9fb223bde7a296d4cc73764355eca"} Jul 06 20:01:07.639360 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:07.639007 2565 scope.go:117] "RemoveContainer" containerID="c5a5ddc37d344ed92b3641c5470364be06bdc0b4d7238ccc814686a337bb88fa" Jul 06 20:01:07.661176 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:07.661144 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn"] Jul 06 20:01:07.665941 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:07.665914 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-62de5f-b5cc98786-85nxn"] Jul 06 20:01:09.204307 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:09.204275 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a194f34-a47a-4c27-8590-0b4ae931b922" path="/var/lib/kubelet/pods/9a194f34-a47a-4c27-8590-0b4ae931b922/volumes" Jul 06 20:01:09.242670 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:09.242640 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 20:01:09.243815 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:09.243794 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 20:01:09.248131 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:09.248109 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 20:01:09.249059 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:09.249041 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 20:01:10.358943 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.358871 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 20:01:10.518683 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.518634 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-token\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.518893 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.518712 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-envoy\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.518893 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.518732 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-data\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.518893 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.518841 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-credential-socket\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.519069 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.518915 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cksgn\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-kube-api-access-cksgn\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.519069 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.518956 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-socket\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.519069 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.519003 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istiod-ca-cert\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.519069 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.519033 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-podinfo\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.519069 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.519066 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-certs\") pod \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\" (UID: \"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff\") " Jul 06 20:01:10.519346 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.519140 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:10.519346 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.519236 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:10.519456 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.519363 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:01:10.519509 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.519465 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:10.520236 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.520154 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:10.520963 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.520501 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-credential-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.520963 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.520529 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.520963 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.520557 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istiod-ca-cert\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.527172 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.527114 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:10.527172 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.527146 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-kube-api-access-cksgn" (OuterVolumeSpecName: "kube-api-access-cksgn") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "kube-api-access-cksgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:01:10.527373 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.527178 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:01:10.527719 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.527699 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" (UID: "dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:01:10.621967 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.621858 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cksgn\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-kube-api-access-cksgn\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.621967 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.621906 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-podinfo\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.621967 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.621917 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-workload-certs\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.621967 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.621926 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-token\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.621967 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.621934 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-envoy\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.621967 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.621942 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff-istio-data\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:10.655447 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.655412 2565 generic.go:356] "Generic (PLEG): container finished" podID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerID="c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9" exitCode=0 Jul 06 20:01:10.655609 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.655457 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" event={"ID":"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff","Type":"ContainerDied","Data":"c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9"} Jul 06 20:01:10.655609 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.655475 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" Jul 06 20:01:10.655609 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.655524 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2" event={"ID":"dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff","Type":"ContainerDied","Data":"ad2cad2db8ba19bc547db671714962105315c549c5370da76397c9f44080eed6"} Jul 06 20:01:10.655609 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.655546 2565 scope.go:117] "RemoveContainer" containerID="c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9" Jul 06 20:01:10.664731 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.664713 2565 scope.go:117] "RemoveContainer" containerID="a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932" Jul 06 20:01:10.673101 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.673079 2565 scope.go:117] "RemoveContainer" containerID="c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9" Jul 06 20:01:10.673480 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:10.673459 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9\": container with ID starting with c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9 not found: ID does not exist" containerID="c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9" Jul 06 20:01:10.673539 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.673491 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9"} err="failed to get container status \"c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9\": rpc error: code = NotFound desc = could not find container \"c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9\": container with ID starting with c98b46c97f8a24b2662f01999b3ffd0954b6c85aa67778617f6708210957cfa9 not found: ID does not exist" Jul 06 20:01:10.673539 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.673512 2565 scope.go:117] "RemoveContainer" containerID="a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932" Jul 06 20:01:10.673806 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:10.673789 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932\": container with ID starting with a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932 not found: ID does not exist" containerID="a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932" Jul 06 20:01:10.673850 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.673813 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932"} err="failed to get container status \"a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932\": rpc error: code = NotFound desc = could not find container \"a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932\": container with ID starting with a2bc167426c00c462e29ba45d860f9f95aca56e9c11ab5b2f07502201cb36932 not found: ID does not exist" Jul 06 20:01:10.696291 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.696254 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2"] Jul 06 20:01:10.710623 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:10.710592 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d188f8-openshift-default-5ff7b448d7-jxrj2"] Jul 06 20:01:11.204178 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:11.204135 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" path="/var/lib/kubelet/pods/dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff/volumes" Jul 06 20:01:14.269669 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.269633 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf"] Jul 06 20:01:14.270057 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.269985 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" Jul 06 20:01:14.270057 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.269997 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" Jul 06 20:01:14.270057 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270004 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" Jul 06 20:01:14.270057 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270009 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" Jul 06 20:01:14.270057 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270017 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4de06a8-972b-45fb-b852-4f46ed56a22a" containerName="maas-api" Jul 06 20:01:14.270057 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270023 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4de06a8-972b-45fb-b852-4f46ed56a22a" containerName="maas-api" Jul 06 20:01:14.270057 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270048 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a194f34-a47a-4c27-8590-0b4ae931b922" containerName="maas-api" Jul 06 20:01:14.270057 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270053 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a194f34-a47a-4c27-8590-0b4ae931b922" containerName="maas-api" Jul 06 20:01:14.270347 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270105 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a194f34-a47a-4c27-8590-0b4ae931b922" containerName="maas-api" Jul 06 20:01:14.270347 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270113 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4de06a8-972b-45fb-b852-4f46ed56a22a" containerName="maas-api" Jul 06 20:01:14.270347 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270121 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" Jul 06 20:01:14.270347 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.270128 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dee3a4bd-68b2-4bd7-a5be-b6f3f21293ff" containerName="istio-proxy" Jul 06 20:01:14.273151 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.273133 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.276078 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.276053 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-efc3ac-openshift-default-dockercfg-578rz\"" Jul 06 20:01:14.289503 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.289470 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf"] Jul 06 20:01:14.360874 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.360838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-podinfo\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.361072 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.360882 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62f5s\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-kube-api-access-62f5s\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.361072 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.361007 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-envoy\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.361072 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.361054 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-certs\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.361232 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.361083 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istiod-ca-cert\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.361232 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.361110 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-data\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.361316 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.361225 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-socket\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.361316 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.361262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-credential-socket\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.361316 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.361308 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-token\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.461746 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.461713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-envoy\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.461946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.461758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-certs\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.461946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.461783 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istiod-ca-cert\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.461946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.461824 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-data\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.461946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.461882 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-socket\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.461946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.461911 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-credential-socket\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.462269 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.462017 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-token\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.462351 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.462305 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-certs\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.462435 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.462405 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-socket\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.462490 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.462452 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-credential-socket\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.462490 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.462463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-podinfo\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.462601 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.462518 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62f5s\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-kube-api-access-62f5s\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.462770 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.462743 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istiod-ca-cert\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.462886 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.462790 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-data\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.465414 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.465392 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-envoy\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.465655 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.465635 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-podinfo\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.474785 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.474760 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-token\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.478086 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.478056 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62f5s\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-kube-api-access-62f5s\") pod \"e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.583521 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.583482 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:14.729115 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.729090 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf"] Jul 06 20:01:14.731193 ip-10-0-143-100 kubenswrapper[2565]: W0706 20:01:14.731172 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f67be2_9d00_4be3_9398_a3d241d20bdc.slice/crio-0e1265266682e527b759f57e81f5a171bdd8aa9f0a12cfc182fbfd13106abdc0 WatchSource:0}: Error finding container 0e1265266682e527b759f57e81f5a171bdd8aa9f0a12cfc182fbfd13106abdc0: Status 404 returned error can't find the container with id 0e1265266682e527b759f57e81f5a171bdd8aa9f0a12cfc182fbfd13106abdc0 Jul 06 20:01:14.733306 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.733271 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:14.733394 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.733356 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:14.733434 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:14.733404 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:15.685253 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:15.685197 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" event={"ID":"a7f67be2-9d00-4be3-9398-a3d241d20bdc","Type":"ContainerStarted","Data":"379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527"} Jul 06 20:01:15.685677 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:15.685276 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" event={"ID":"a7f67be2-9d00-4be3-9398-a3d241d20bdc","Type":"ContainerStarted","Data":"0e1265266682e527b759f57e81f5a171bdd8aa9f0a12cfc182fbfd13106abdc0"} Jul 06 20:01:15.714937 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:15.714871 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" podStartSLOduration=1.7148548909999999 podStartE2EDuration="1.714854891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:15.71303537 +0000 UTC m=+2107.087491310" watchObservedRunningTime="2026-07-06 20:01:15.714854891 +0000 UTC m=+2107.089310830" Jul 06 20:01:16.584019 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:16.583975 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:16.589090 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:16.589060 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:16.690155 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:16.690128 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:16.691070 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:16.691052 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:18.080855 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.080822 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7"] Jul 06 20:01:18.084448 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.084424 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.087741 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.087722 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-efc3ac-serving-cert\"" Jul 06 20:01:18.095476 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.095446 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7"] Jul 06 20:01:18.203069 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.203034 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97d10d4-73d1-4245-a6d9-66f74950e977-maas-api-tls\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.203316 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.203103 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97d10d4-73d1-4245-a6d9-66f74950e977-openshift-service-ca\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.203316 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.203175 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pchdv\" (UniqueName: \"kubernetes.io/projected/a97d10d4-73d1-4245-a6d9-66f74950e977-kube-api-access-pchdv\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.304044 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.303998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97d10d4-73d1-4245-a6d9-66f74950e977-maas-api-tls\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.304044 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.304052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97d10d4-73d1-4245-a6d9-66f74950e977-openshift-service-ca\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.304308 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.304096 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pchdv\" (UniqueName: \"kubernetes.io/projected/a97d10d4-73d1-4245-a6d9-66f74950e977-kube-api-access-pchdv\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.304792 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.304769 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97d10d4-73d1-4245-a6d9-66f74950e977-openshift-service-ca\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.306622 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.306594 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97d10d4-73d1-4245-a6d9-66f74950e977-maas-api-tls\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.315947 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.315918 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pchdv\" (UniqueName: \"kubernetes.io/projected/a97d10d4-73d1-4245-a6d9-66f74950e977-kube-api-access-pchdv\") pod \"maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.395487 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.395392 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:18.542742 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.542717 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7"] Jul 06 20:01:18.545036 ip-10-0-143-100 kubenswrapper[2565]: W0706 20:01:18.545004 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97d10d4_73d1_4245_a6d9_66f74950e977.slice/crio-00ccac8e9ccd388a20d9c0692f30ecbf88693e3751e87b777425d1e2995a66a0 WatchSource:0}: Error finding container 00ccac8e9ccd388a20d9c0692f30ecbf88693e3751e87b777425d1e2995a66a0: Status 404 returned error can't find the container with id 00ccac8e9ccd388a20d9c0692f30ecbf88693e3751e87b777425d1e2995a66a0 Jul 06 20:01:18.699962 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:18.699875 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" event={"ID":"a97d10d4-73d1-4245-a6d9-66f74950e977","Type":"ContainerStarted","Data":"00ccac8e9ccd388a20d9c0692f30ecbf88693e3751e87b777425d1e2995a66a0"} Jul 06 20:01:19.706102 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:19.706068 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" event={"ID":"a97d10d4-73d1-4245-a6d9-66f74950e977","Type":"ContainerStarted","Data":"16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14"} Jul 06 20:01:19.706602 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:19.706146 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:25.722642 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:25.722586 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:25.747719 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:25.747662 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" podStartSLOduration=7.097607203 podStartE2EDuration="7.747646639s" totalImagesPullingTime="650.039436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:19.734714822 +0000 UTC m=+2111.109170772" watchObservedRunningTime="2026-07-06 20:01:25.747646639 +0000 UTC m=+2117.122102578" Jul 06 20:01:26.527740 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.527674 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w"] Jul 06 20:01:26.535778 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.535751 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:26.541635 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.541608 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-e82419-serving-cert\"" Jul 06 20:01:26.544839 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.544809 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w"] Jul 06 20:01:26.569191 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.569163 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:26.569362 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.569314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-openshift-service-ca\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:26.569362 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.569344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2chc\" (UniqueName: \"kubernetes.io/projected/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-kube-api-access-w2chc\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:26.670160 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.670128 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-openshift-service-ca\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:26.670160 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.670162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2chc\" (UniqueName: \"kubernetes.io/projected/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-kube-api-access-w2chc\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:26.670473 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.670264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:26.670473 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:26.670374 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-e82419-serving-cert: secret "maas-api-e2e-api-b-e82419-serving-cert" not found Jul 06 20:01:26.670473 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:26.670431 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls podName:f13f0816-b5cd-429a-8ad8-4fe6d14aa88a nodeName:}" failed. No retries permitted until 2026-07-06 20:01:27.170415871 +0000 UTC m=+2118.544871790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls") pod "maas-api-e2e-api-b-e82419-66cc769754-sw56w" (UID: "f13f0816-b5cd-429a-8ad8-4fe6d14aa88a") : secret "maas-api-e2e-api-b-e82419-serving-cert" not found Jul 06 20:01:26.670963 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.670939 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-openshift-service-ca\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:26.683537 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:26.683509 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2chc\" (UniqueName: \"kubernetes.io/projected/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-kube-api-access-w2chc\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:27.173967 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:27.173919 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:27.176485 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:27.176457 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls\") pod \"maas-api-e2e-api-b-e82419-66cc769754-sw56w\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:27.454387 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:27.454275 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:27.608851 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:27.608821 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w"] Jul 06 20:01:27.611669 ip-10-0-143-100 kubenswrapper[2565]: W0706 20:01:27.611635 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13f0816_b5cd_429a_8ad8_4fe6d14aa88a.slice/crio-23ebe91f7b1564df7ac164d4ce461c7274a3ff321434661b41427bfc0d41c7bf WatchSource:0}: Error finding container 23ebe91f7b1564df7ac164d4ce461c7274a3ff321434661b41427bfc0d41c7bf: Status 404 returned error can't find the container with id 23ebe91f7b1564df7ac164d4ce461c7274a3ff321434661b41427bfc0d41c7bf Jul 06 20:01:27.753617 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:27.753531 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" event={"ID":"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a","Type":"ContainerStarted","Data":"23ebe91f7b1564df7ac164d4ce461c7274a3ff321434661b41427bfc0d41c7bf"} Jul 06 20:01:28.760995 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:28.760957 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" event={"ID":"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a","Type":"ContainerStarted","Data":"bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f"} Jul 06 20:01:28.761450 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:28.761088 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:28.783418 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:28.783364 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" podStartSLOduration=2.2694468629999998 podStartE2EDuration="2.783345937s" totalImagesPullingTime="513.899074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:28.780427791 +0000 UTC m=+2120.154883731" watchObservedRunningTime="2026-07-06 20:01:28.783345937 +0000 UTC m=+2120.157801922" Jul 06 20:01:29.767116 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:29.767081 2565 generic.go:356] "Generic (PLEG): container finished" podID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerID="379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527" exitCode=0 Jul 06 20:01:29.767671 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:29.767155 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" event={"ID":"a7f67be2-9d00-4be3-9398-a3d241d20bdc","Type":"ContainerDied","Data":"379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527"} Jul 06 20:01:29.767742 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:29.767717 2565 scope.go:117] "RemoveContainer" containerID="379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527" Jul 06 20:01:29.771512 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:29.771481 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:29.771626 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:29.771551 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:29.771626 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:29.771582 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:30.773161 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:30.773126 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" event={"ID":"a7f67be2-9d00-4be3-9398-a3d241d20bdc","Type":"ContainerStarted","Data":"dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0"} Jul 06 20:01:31.584248 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:31.584192 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:31.585460 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:31.585430 2565 patch_prober.go:28] interesting pod/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 06 20:01:31.585600 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:31.585488 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 06 20:01:31.690567 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:31.690532 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:32.584918 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:32.584874 2565 patch_prober.go:28] interesting pod/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 06 20:01:32.585448 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:32.585241 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 06 20:01:33.584306 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:33.584270 2565 patch_prober.go:28] interesting pod/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 06 20:01:33.584513 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:33.584330 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 06 20:01:34.584066 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:34.584030 2565 patch_prober.go:28] interesting pod/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 06 20:01:34.584462 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:34.584083 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 06 20:01:34.772806 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:34.772780 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:35.588336 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:35.588301 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:35.589437 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:35.589414 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:40.184033 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.183996 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7"] Jul 06 20:01:40.184605 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.184335 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" podUID="a97d10d4-73d1-4245-a6d9-66f74950e977" containerName="maas-api" containerID="cri-o://16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14" gracePeriod=30 Jul 06 20:01:40.436005 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.435946 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:40.502138 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.502107 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97d10d4-73d1-4245-a6d9-66f74950e977-maas-api-tls\") pod \"a97d10d4-73d1-4245-a6d9-66f74950e977\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " Jul 06 20:01:40.502325 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.502149 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pchdv\" (UniqueName: \"kubernetes.io/projected/a97d10d4-73d1-4245-a6d9-66f74950e977-kube-api-access-pchdv\") pod \"a97d10d4-73d1-4245-a6d9-66f74950e977\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " Jul 06 20:01:40.502325 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.502220 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97d10d4-73d1-4245-a6d9-66f74950e977-openshift-service-ca\") pod \"a97d10d4-73d1-4245-a6d9-66f74950e977\" (UID: \"a97d10d4-73d1-4245-a6d9-66f74950e977\") " Jul 06 20:01:40.502617 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.502592 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a97d10d4-73d1-4245-a6d9-66f74950e977-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a97d10d4-73d1-4245-a6d9-66f74950e977" (UID: "a97d10d4-73d1-4245-a6d9-66f74950e977"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:01:40.504301 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.504280 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a97d10d4-73d1-4245-a6d9-66f74950e977-kube-api-access-pchdv" (OuterVolumeSpecName: "kube-api-access-pchdv") pod "a97d10d4-73d1-4245-a6d9-66f74950e977" (UID: "a97d10d4-73d1-4245-a6d9-66f74950e977"). InnerVolumeSpecName "kube-api-access-pchdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:01:40.504372 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.504277 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a97d10d4-73d1-4245-a6d9-66f74950e977-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a97d10d4-73d1-4245-a6d9-66f74950e977" (UID: "a97d10d4-73d1-4245-a6d9-66f74950e977"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:01:40.603452 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.603416 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97d10d4-73d1-4245-a6d9-66f74950e977-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:40.603452 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.603445 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97d10d4-73d1-4245-a6d9-66f74950e977-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:40.603452 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.603455 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pchdv\" (UniqueName: \"kubernetes.io/projected/a97d10d4-73d1-4245-a6d9-66f74950e977-kube-api-access-pchdv\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:40.821890 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.821859 2565 generic.go:356] "Generic (PLEG): container finished" podID="a97d10d4-73d1-4245-a6d9-66f74950e977" containerID="16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14" exitCode=0 Jul 06 20:01:40.822144 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.821911 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" Jul 06 20:01:40.822144 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.821959 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" event={"ID":"a97d10d4-73d1-4245-a6d9-66f74950e977","Type":"ContainerDied","Data":"16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14"} Jul 06 20:01:40.822144 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.822012 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7" event={"ID":"a97d10d4-73d1-4245-a6d9-66f74950e977","Type":"ContainerDied","Data":"00ccac8e9ccd388a20d9c0692f30ecbf88693e3751e87b777425d1e2995a66a0"} Jul 06 20:01:40.822144 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.822036 2565 scope.go:117] "RemoveContainer" containerID="16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14" Jul 06 20:01:40.830968 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.830951 2565 scope.go:117] "RemoveContainer" containerID="16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14" Jul 06 20:01:40.831234 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:40.831190 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14\": container with ID starting with 16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14 not found: ID does not exist" containerID="16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14" Jul 06 20:01:40.831304 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.831235 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14"} err="failed to get container status \"16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14\": rpc error: code = NotFound desc = could not find container \"16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14\": container with ID starting with 16758abfea0847367059de0af62c22284fc9e409155b867369633d2893e81b14 not found: ID does not exist" Jul 06 20:01:40.846261 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.846233 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7"] Jul 06 20:01:40.851070 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:40.851047 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-efc3ac-5fc86f7cdc-zgtd7"] Jul 06 20:01:41.204349 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:41.204256 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a97d10d4-73d1-4245-a6d9-66f74950e977" path="/var/lib/kubelet/pods/a97d10d4-73d1-4245-a6d9-66f74950e977/volumes" Jul 06 20:01:46.261023 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:46.260985 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf"] Jul 06 20:01:46.261503 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:46.261261 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" containerID="cri-o://dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0" gracePeriod=30 Jul 06 20:01:47.534388 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.534351 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w"] Jul 06 20:01:47.534874 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.534543 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" podUID="f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" containerName="maas-api" containerID="cri-o://bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f" gracePeriod=30 Jul 06 20:01:47.778601 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.778573 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:47.857997 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.857962 2565 generic.go:356] "Generic (PLEG): container finished" podID="f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" containerID="bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f" exitCode=0 Jul 06 20:01:47.858189 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.858006 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" Jul 06 20:01:47.858189 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.858054 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" event={"ID":"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a","Type":"ContainerDied","Data":"bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f"} Jul 06 20:01:47.858189 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.858103 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w" event={"ID":"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a","Type":"ContainerDied","Data":"23ebe91f7b1564df7ac164d4ce461c7274a3ff321434661b41427bfc0d41c7bf"} Jul 06 20:01:47.858189 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.858126 2565 scope.go:117] "RemoveContainer" containerID="bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f" Jul 06 20:01:47.866977 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.866959 2565 scope.go:117] "RemoveContainer" containerID="bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f" Jul 06 20:01:47.867259 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:47.867242 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f\": container with ID starting with bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f not found: ID does not exist" containerID="bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f" Jul 06 20:01:47.867313 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.867268 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f"} err="failed to get container status \"bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f\": rpc error: code = NotFound desc = could not find container \"bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f\": container with ID starting with bef5ae8eb68d3a060f91f3d584918a11a898ed642aee0fe59aa7db0fa7f6cc3f not found: ID does not exist" Jul 06 20:01:47.876569 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.876545 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls\") pod \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " Jul 06 20:01:47.876671 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.876660 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2chc\" (UniqueName: \"kubernetes.io/projected/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-kube-api-access-w2chc\") pod \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " Jul 06 20:01:47.876729 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.876717 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-openshift-service-ca\") pod \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\" (UID: \"f13f0816-b5cd-429a-8ad8-4fe6d14aa88a\") " Jul 06 20:01:47.877048 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.877024 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" (UID: "f13f0816-b5cd-429a-8ad8-4fe6d14aa88a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:01:47.878606 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.878585 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" (UID: "f13f0816-b5cd-429a-8ad8-4fe6d14aa88a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:01:47.878696 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.878638 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-kube-api-access-w2chc" (OuterVolumeSpecName: "kube-api-access-w2chc") pod "f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" (UID: "f13f0816-b5cd-429a-8ad8-4fe6d14aa88a"). InnerVolumeSpecName "kube-api-access-w2chc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:01:47.977397 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.977354 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:47.977397 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.977390 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:47.977397 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:47.977404 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2chc\" (UniqueName: \"kubernetes.io/projected/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a-kube-api-access-w2chc\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:48.192697 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:48.192665 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w"] Jul 06 20:01:48.199525 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:48.199483 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e82419-66cc769754-sw56w"] Jul 06 20:01:49.204289 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:49.204256 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" path="/var/lib/kubelet/pods/f13f0816-b5cd-429a-8ad8-4fe6d14aa88a/volumes" Jul 06 20:01:50.590987 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:50.590945 2565 patch_prober.go:28] interesting pod/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 20:01:50.591517 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:50.591033 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:01:51.533894 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.533824 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:51.613433 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613403 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62f5s\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-kube-api-access-62f5s\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613456 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-certs\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613476 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-credential-socket\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613504 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-data\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613529 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-token\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613556 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-envoy\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613588 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-podinfo\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613646 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istiod-ca-cert\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613674 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-socket\") pod \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\" (UID: \"a7f67be2-9d00-4be3-9398-a3d241d20bdc\") " Jul 06 20:01:51.613895 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613702 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:51.614438 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613948 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-certs\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.614438 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.613945 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:51.614438 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.614119 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:51.614438 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.614140 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:01:51.614664 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.614490 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:51.615799 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.615764 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:01:51.615902 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.615801 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-kube-api-access-62f5s" (OuterVolumeSpecName: "kube-api-access-62f5s") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "kube-api-access-62f5s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:01:51.615902 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.615797 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:01:51.616032 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.616008 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7f67be2-9d00-4be3-9398-a3d241d20bdc" (UID: "a7f67be2-9d00-4be3-9398-a3d241d20bdc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:01:51.714689 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.714658 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-envoy\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.714689 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.714685 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-podinfo\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.714689 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.714694 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istiod-ca-cert\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.714946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.714703 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-workload-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.714946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.714714 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62f5s\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-kube-api-access-62f5s\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.714946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.714724 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-credential-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.714946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.714732 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-data\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.714946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.714740 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f67be2-9d00-4be3-9398-a3d241d20bdc-istio-token\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:01:51.879455 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.879422 2565 generic.go:356] "Generic (PLEG): container finished" podID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerID="dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0" exitCode=0 Jul 06 20:01:51.879628 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.879479 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" Jul 06 20:01:51.879628 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.879508 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" event={"ID":"a7f67be2-9d00-4be3-9398-a3d241d20bdc","Type":"ContainerDied","Data":"dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0"} Jul 06 20:01:51.879628 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.879561 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf" event={"ID":"a7f67be2-9d00-4be3-9398-a3d241d20bdc","Type":"ContainerDied","Data":"0e1265266682e527b759f57e81f5a171bdd8aa9f0a12cfc182fbfd13106abdc0"} Jul 06 20:01:51.879628 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.879584 2565 scope.go:117] "RemoveContainer" containerID="dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0" Jul 06 20:01:51.888871 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.888852 2565 scope.go:117] "RemoveContainer" containerID="379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527" Jul 06 20:01:51.896589 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.896570 2565 scope.go:117] "RemoveContainer" containerID="dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0" Jul 06 20:01:51.896847 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:51.896826 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0\": container with ID starting with dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0 not found: ID does not exist" containerID="dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0" Jul 06 20:01:51.896933 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.896862 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0"} err="failed to get container status \"dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0\": rpc error: code = NotFound desc = could not find container \"dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0\": container with ID starting with dd3d06415053916e3ac2371bd69bd0e761a86ce718ced15fb4f3560fac1790b0 not found: ID does not exist" Jul 06 20:01:51.896933 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.896887 2565 scope.go:117] "RemoveContainer" containerID="379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527" Jul 06 20:01:51.897129 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:51.897112 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527\": container with ID starting with 379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527 not found: ID does not exist" containerID="379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527" Jul 06 20:01:51.897169 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.897137 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527"} err="failed to get container status \"379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527\": rpc error: code = NotFound desc = could not find container \"379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527\": container with ID starting with 379ef697fe7560cac231959c82d8bc0b667bfe7ebb3a71a249ff5202e3898527 not found: ID does not exist" Jul 06 20:01:51.906400 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.906368 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf"] Jul 06 20:01:51.914523 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:51.914496 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-efc3ac-openshift-default-7ddcf489d9-25qmf"] Jul 06 20:01:53.204398 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:53.204369 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" path="/var/lib/kubelet/pods/a7f67be2-9d00-4be3-9398-a3d241d20bdc/volumes" Jul 06 20:01:55.097909 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.097875 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc"] Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098280 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098294 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098304 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098320 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098339 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a97d10d4-73d1-4245-a6d9-66f74950e977" containerName="maas-api" Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098344 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a97d10d4-73d1-4245-a6d9-66f74950e977" containerName="maas-api" Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098351 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" containerName="maas-api" Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098356 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" containerName="maas-api" Jul 06 20:01:55.098411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098417 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" Jul 06 20:01:55.098900 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098429 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13f0816-b5cd-429a-8ad8-4fe6d14aa88a" containerName="maas-api" Jul 06 20:01:55.098900 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.098437 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a97d10d4-73d1-4245-a6d9-66f74950e977" containerName="maas-api" Jul 06 20:01:55.102950 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.102931 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.106501 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.106476 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-cd90ec-openshift-default-dockercfg-sqx8h\"" Jul 06 20:01:55.124379 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.124354 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc"] Jul 06 20:01:55.248483 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248444 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-envoy\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.248483 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248481 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz7nr\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-kube-api-access-lz7nr\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.248727 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248543 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-certs\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.248727 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248561 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-socket\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.248727 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istiod-ca-cert\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.248727 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248684 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-data\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.248727 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-credential-socket\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.248905 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-podinfo\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.248905 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.248807 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-token\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.349899 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.349816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-certs\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.349899 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.349851 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-socket\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.349899 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.349868 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istiod-ca-cert\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.349899 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.349890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-data\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350332 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350021 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-credential-socket\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350332 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350063 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-podinfo\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350332 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-token\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350332 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-envoy\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350332 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350246 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz7nr\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-kube-api-access-lz7nr\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350592 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350337 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-socket\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350592 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350349 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-certs\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350592 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350447 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-data\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350727 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350701 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istiod-ca-cert\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.350862 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.350765 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-credential-socket\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.352471 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.352449 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-envoy\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.352784 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.352766 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-podinfo\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.360851 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.360828 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-token\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.360983 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.360967 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz7nr\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-kube-api-access-lz7nr\") pod \"e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.413478 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.413444 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:55.552830 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.552803 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc"] Jul 06 20:01:55.554512 ip-10-0-143-100 kubenswrapper[2565]: W0706 20:01:55.554482 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeec3cb2_c49d_48a9_bedb_49a7c696cfa4.slice/crio-1c51d9c15dbde320a3b20e14d31f1feedb05463be36e174e6940624680826dc2 WatchSource:0}: Error finding container 1c51d9c15dbde320a3b20e14d31f1feedb05463be36e174e6940624680826dc2: Status 404 returned error can't find the container with id 1c51d9c15dbde320a3b20e14d31f1feedb05463be36e174e6940624680826dc2 Jul 06 20:01:55.556599 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.556558 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:55.556683 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.556655 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:55.556722 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.556709 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:01:55.901845 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.901755 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" event={"ID":"beec3cb2-c49d-48a9-bedb-49a7c696cfa4","Type":"ContainerStarted","Data":"7d8aff1b6ee5626e7a034e96886afedb276f2e1e61e6381060ebaf8a16ac9f58"} Jul 06 20:01:55.901845 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.901813 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" event={"ID":"beec3cb2-c49d-48a9-bedb-49a7c696cfa4","Type":"ContainerStarted","Data":"1c51d9c15dbde320a3b20e14d31f1feedb05463be36e174e6940624680826dc2"} Jul 06 20:01:55.927993 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:55.927936 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" podStartSLOduration=0.927919875 podStartE2EDuration="927.919875ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:55.926010632 +0000 UTC m=+2147.300466573" watchObservedRunningTime="2026-07-06 20:01:55.927919875 +0000 UTC m=+2147.302375814" Jul 06 20:01:56.414385 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:56.414347 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:56.419722 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:56.419699 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:56.906758 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:56.906729 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:56.907742 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:56.907719 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:01:58.003766 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.003728 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8"] Jul 06 20:01:58.004345 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.004326 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f67be2-9d00-4be3-9398-a3d241d20bdc" containerName="istio-proxy" Jul 06 20:01:58.007378 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.007356 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.010272 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.010254 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-cd90ec-serving-cert\"" Jul 06 20:01:58.017638 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.017614 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8"] Jul 06 20:01:58.175738 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.175700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.175920 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.175749 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtknk\" (UniqueName: \"kubernetes.io/projected/6f7a25b2-8879-4072-83a3-779637d0963b-kube-api-access-gtknk\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.175920 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.175779 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7a25b2-8879-4072-83a3-779637d0963b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.276721 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.276619 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.276721 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.276689 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtknk\" (UniqueName: \"kubernetes.io/projected/6f7a25b2-8879-4072-83a3-779637d0963b-kube-api-access-gtknk\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.276973 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.276725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7a25b2-8879-4072-83a3-779637d0963b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.276973 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:58.276770 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-cd90ec-serving-cert: secret "maas-api-e2e-shared-a-cd90ec-serving-cert" not found Jul 06 20:01:58.276973 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:01:58.276887 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls podName:6f7a25b2-8879-4072-83a3-779637d0963b nodeName:}" failed. No retries permitted until 2026-07-06 20:01:58.776865328 +0000 UTC m=+2150.151321260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls") pod "maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" (UID: "6f7a25b2-8879-4072-83a3-779637d0963b") : secret "maas-api-e2e-shared-a-cd90ec-serving-cert" not found Jul 06 20:01:58.277496 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.277472 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7a25b2-8879-4072-83a3-779637d0963b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.287157 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.287134 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtknk\" (UniqueName: \"kubernetes.io/projected/6f7a25b2-8879-4072-83a3-779637d0963b-kube-api-access-gtknk\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.781353 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.781321 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.783822 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.783792 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls\") pod \"maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:58.919737 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:58.919705 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:59.058769 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:59.058742 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8"] Jul 06 20:01:59.060734 ip-10-0-143-100 kubenswrapper[2565]: W0706 20:01:59.060710 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7a25b2_8879_4072_83a3_779637d0963b.slice/crio-9c4163151fb3f9e95715ef95feced32e1ecfbd02e7a1228adb3c63eeec70879f WatchSource:0}: Error finding container 9c4163151fb3f9e95715ef95feced32e1ecfbd02e7a1228adb3c63eeec70879f: Status 404 returned error can't find the container with id 9c4163151fb3f9e95715ef95feced32e1ecfbd02e7a1228adb3c63eeec70879f Jul 06 20:01:59.922447 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:59.922411 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" event={"ID":"6f7a25b2-8879-4072-83a3-779637d0963b","Type":"ContainerStarted","Data":"4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84"} Jul 06 20:01:59.922672 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:59.922464 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" event={"ID":"6f7a25b2-8879-4072-83a3-779637d0963b","Type":"ContainerStarted","Data":"9c4163151fb3f9e95715ef95feced32e1ecfbd02e7a1228adb3c63eeec70879f"} Jul 06 20:01:59.922672 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:59.922545 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:01:59.945897 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:01:59.945838 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" podStartSLOduration=2.475385883 podStartE2EDuration="2.945821574s" totalImagesPullingTime="470.435691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:59.942939996 +0000 UTC m=+2151.317395937" watchObservedRunningTime="2026-07-06 20:01:59.945821574 +0000 UTC m=+2151.320277513" Jul 06 20:02:05.938514 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:05.938429 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:02:06.298325 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.298207 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9"] Jul 06 20:02:06.302752 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.302721 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:06.306226 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.306190 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-04d79c-serving-cert\"" Jul 06 20:02:06.340922 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.340874 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9"] Jul 06 20:02:06.461655 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.461612 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:06.461827 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.461694 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppfjt\" (UniqueName: \"kubernetes.io/projected/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-kube-api-access-ppfjt\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:06.461827 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.461742 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-openshift-service-ca\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:06.562809 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.562717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:06.562962 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.562808 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppfjt\" (UniqueName: \"kubernetes.io/projected/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-kube-api-access-ppfjt\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:06.562962 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.562874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-openshift-service-ca\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:06.562962 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:02:06.562878 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-04d79c-serving-cert: secret "maas-api-e2e-shared-b-04d79c-serving-cert" not found Jul 06 20:02:06.563090 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:02:06.563040 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls podName:3fe4e6d6-0278-45cd-bb4b-49f96f449b85 nodeName:}" failed. No retries permitted until 2026-07-06 20:02:07.063020179 +0000 UTC m=+2158.437476102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls") pod "maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" (UID: "3fe4e6d6-0278-45cd-bb4b-49f96f449b85") : secret "maas-api-e2e-shared-b-04d79c-serving-cert" not found Jul 06 20:02:06.563494 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.563475 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-openshift-service-ca\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:06.580506 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:06.580484 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppfjt\" (UniqueName: \"kubernetes.io/projected/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-kube-api-access-ppfjt\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:07.068960 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:07.068927 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:07.071422 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:07.071395 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls\") pod \"maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:07.216237 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:07.213895 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:07.392372 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:07.392343 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9"] Jul 06 20:02:07.394777 ip-10-0-143-100 kubenswrapper[2565]: W0706 20:02:07.394747 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe4e6d6_0278_45cd_bb4b_49f96f449b85.slice/crio-f43f73a5583a8dd16854de083dd514b2fddd3d738534479b1289752b9af6fefd WatchSource:0}: Error finding container f43f73a5583a8dd16854de083dd514b2fddd3d738534479b1289752b9af6fefd: Status 404 returned error can't find the container with id f43f73a5583a8dd16854de083dd514b2fddd3d738534479b1289752b9af6fefd Jul 06 20:02:07.960975 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:07.960943 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" event={"ID":"3fe4e6d6-0278-45cd-bb4b-49f96f449b85","Type":"ContainerStarted","Data":"f43f73a5583a8dd16854de083dd514b2fddd3d738534479b1289752b9af6fefd"} Jul 06 20:02:08.967649 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:08.967609 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" event={"ID":"3fe4e6d6-0278-45cd-bb4b-49f96f449b85","Type":"ContainerStarted","Data":"6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515"} Jul 06 20:02:08.968072 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:08.967758 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:08.990484 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:08.990430 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" podStartSLOduration=2.386452726 podStartE2EDuration="2.990412468s" totalImagesPullingTime="603.959742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:02:08.987807628 +0000 UTC m=+2160.362263584" watchObservedRunningTime="2026-07-06 20:02:08.990412468 +0000 UTC m=+2160.364868448" Jul 06 20:02:10.978698 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:10.978665 2565 generic.go:356] "Generic (PLEG): container finished" podID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerID="7d8aff1b6ee5626e7a034e96886afedb276f2e1e61e6381060ebaf8a16ac9f58" exitCode=0 Jul 06 20:02:10.979139 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:10.978743 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" event={"ID":"beec3cb2-c49d-48a9-bedb-49a7c696cfa4","Type":"ContainerDied","Data":"7d8aff1b6ee5626e7a034e96886afedb276f2e1e61e6381060ebaf8a16ac9f58"} Jul 06 20:02:10.979202 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:10.979141 2565 scope.go:117] "RemoveContainer" containerID="7d8aff1b6ee5626e7a034e96886afedb276f2e1e61e6381060ebaf8a16ac9f58" Jul 06 20:02:10.980288 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:10.980257 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:02:10.980382 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:10.980345 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:02:10.980423 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:10.980385 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:02:11.907455 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:11.907418 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:02:11.984854 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:11.984817 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" event={"ID":"beec3cb2-c49d-48a9-bedb-49a7c696cfa4","Type":"ContainerStarted","Data":"ad48145e676da840778b5e0faa713f0d1b102e93cab5f2c7852e155825250522"} Jul 06 20:02:12.415022 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:12.413892 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:02:12.415418 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:12.415385 2565 patch_prober.go:28] interesting pod/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 06 20:02:12.415635 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:12.415607 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 06 20:02:13.414319 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:13.414266 2565 patch_prober.go:28] interesting pod/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 06 20:02:13.414843 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:13.414343 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 06 20:02:14.414429 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:14.414388 2565 patch_prober.go:28] interesting pod/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 06 20:02:14.415037 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:14.415002 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 06 20:02:14.978418 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:14.978384 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:02:15.413918 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:15.413881 2565 patch_prober.go:28] interesting pod/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 06 20:02:15.414167 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:15.414141 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 06 20:02:16.417541 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:16.417505 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:02:16.417995 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:16.417781 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:02:16.418490 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:02:16.418474 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:06:09.271340 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:06:09.271309 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 20:06:09.272543 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:06:09.272518 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 20:06:09.276582 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:06:09.276556 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 20:06:09.277632 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:06:09.277615 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ls98_805cce93-10de-4cb9-a20e-29561db3c966/ovn-acl-logging/0.log" Jul 06 20:07:58.590042 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.590010 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8"] Jul 06 20:07:58.590572 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.590202 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" podUID="6f7a25b2-8879-4072-83a3-779637d0963b" containerName="maas-api" containerID="cri-o://4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84" gracePeriod=30 Jul 06 20:07:58.841045 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.840975 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:07:58.958553 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.958516 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7a25b2-8879-4072-83a3-779637d0963b-openshift-service-ca\") pod \"6f7a25b2-8879-4072-83a3-779637d0963b\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " Jul 06 20:07:58.958745 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.958581 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtknk\" (UniqueName: \"kubernetes.io/projected/6f7a25b2-8879-4072-83a3-779637d0963b-kube-api-access-gtknk\") pod \"6f7a25b2-8879-4072-83a3-779637d0963b\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " Jul 06 20:07:58.958745 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.958631 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls\") pod \"6f7a25b2-8879-4072-83a3-779637d0963b\" (UID: \"6f7a25b2-8879-4072-83a3-779637d0963b\") " Jul 06 20:07:58.958949 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.958928 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f7a25b2-8879-4072-83a3-779637d0963b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f7a25b2-8879-4072-83a3-779637d0963b" (UID: "6f7a25b2-8879-4072-83a3-779637d0963b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:07:58.960695 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.960664 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f7a25b2-8879-4072-83a3-779637d0963b" (UID: "6f7a25b2-8879-4072-83a3-779637d0963b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:07:58.960817 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:58.960752 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7a25b2-8879-4072-83a3-779637d0963b-kube-api-access-gtknk" (OuterVolumeSpecName: "kube-api-access-gtknk") pod "6f7a25b2-8879-4072-83a3-779637d0963b" (UID: "6f7a25b2-8879-4072-83a3-779637d0963b"). InnerVolumeSpecName "kube-api-access-gtknk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:07:59.059860 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.059818 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7a25b2-8879-4072-83a3-779637d0963b-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:07:59.059860 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.059851 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtknk\" (UniqueName: \"kubernetes.io/projected/6f7a25b2-8879-4072-83a3-779637d0963b-kube-api-access-gtknk\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:07:59.059860 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.059861 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7a25b2-8879-4072-83a3-779637d0963b-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:07:59.632774 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.632740 2565 generic.go:356] "Generic (PLEG): container finished" podID="6f7a25b2-8879-4072-83a3-779637d0963b" containerID="4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84" exitCode=0 Jul 06 20:07:59.633202 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.632797 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" event={"ID":"6f7a25b2-8879-4072-83a3-779637d0963b","Type":"ContainerDied","Data":"4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84"} Jul 06 20:07:59.633202 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.632800 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" Jul 06 20:07:59.633202 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.632867 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8" event={"ID":"6f7a25b2-8879-4072-83a3-779637d0963b","Type":"ContainerDied","Data":"9c4163151fb3f9e95715ef95feced32e1ecfbd02e7a1228adb3c63eeec70879f"} Jul 06 20:07:59.633202 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.632889 2565 scope.go:117] "RemoveContainer" containerID="4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84" Jul 06 20:07:59.641620 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.641600 2565 scope.go:117] "RemoveContainer" containerID="4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84" Jul 06 20:07:59.641928 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:07:59.641909 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84\": container with ID starting with 4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84 not found: ID does not exist" containerID="4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84" Jul 06 20:07:59.641988 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.641938 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84"} err="failed to get container status \"4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84\": rpc error: code = NotFound desc = could not find container \"4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84\": container with ID starting with 4df9a6ae14cf0adebb7b107a77f97127a20f6045deb42ac261e3f4fbad108c84 not found: ID does not exist" Jul 06 20:07:59.672847 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.672816 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8"] Jul 06 20:07:59.683292 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:07:59.683262 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cd90ec-6dfdb9bd45-sgzp8"] Jul 06 20:08:01.204839 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:01.204804 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7a25b2-8879-4072-83a3-779637d0963b" path="/var/lib/kubelet/pods/6f7a25b2-8879-4072-83a3-779637d0963b/volumes" Jul 06 20:08:41.677407 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:41.677371 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc"] Jul 06 20:08:41.678088 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:41.677620 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" containerID="cri-o://ad48145e676da840778b5e0faa713f0d1b102e93cab5f2c7852e155825250522" gracePeriod=30 Jul 06 20:08:43.113448 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.113411 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9"] Jul 06 20:08:43.113933 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.113670 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" podUID="3fe4e6d6-0278-45cd-bb4b-49f96f449b85" containerName="maas-api" containerID="cri-o://6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515" gracePeriod=30 Jul 06 20:08:43.363758 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.363690 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:08:43.489627 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.489597 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-openshift-service-ca\") pod \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " Jul 06 20:08:43.489816 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.489633 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppfjt\" (UniqueName: \"kubernetes.io/projected/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-kube-api-access-ppfjt\") pod \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " Jul 06 20:08:43.489816 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.489710 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls\") pod \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\" (UID: \"3fe4e6d6-0278-45cd-bb4b-49f96f449b85\") " Jul 06 20:08:43.490038 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.490002 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3fe4e6d6-0278-45cd-bb4b-49f96f449b85" (UID: "3fe4e6d6-0278-45cd-bb4b-49f96f449b85"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:08:43.490175 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.490082 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-openshift-service-ca\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:43.491755 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.491729 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3fe4e6d6-0278-45cd-bb4b-49f96f449b85" (UID: "3fe4e6d6-0278-45cd-bb4b-49f96f449b85"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:08:43.491861 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.491836 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-kube-api-access-ppfjt" (OuterVolumeSpecName: "kube-api-access-ppfjt") pod "3fe4e6d6-0278-45cd-bb4b-49f96f449b85" (UID: "3fe4e6d6-0278-45cd-bb4b-49f96f449b85"). InnerVolumeSpecName "kube-api-access-ppfjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:08:43.590627 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.590590 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-maas-api-tls\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:43.590627 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.590617 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppfjt\" (UniqueName: \"kubernetes.io/projected/3fe4e6d6-0278-45cd-bb4b-49f96f449b85-kube-api-access-ppfjt\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:43.847943 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.847906 2565 generic.go:356] "Generic (PLEG): container finished" podID="3fe4e6d6-0278-45cd-bb4b-49f96f449b85" containerID="6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515" exitCode=0 Jul 06 20:08:43.848118 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.847959 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" Jul 06 20:08:43.848118 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.847993 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" event={"ID":"3fe4e6d6-0278-45cd-bb4b-49f96f449b85","Type":"ContainerDied","Data":"6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515"} Jul 06 20:08:43.848118 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.848049 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9" event={"ID":"3fe4e6d6-0278-45cd-bb4b-49f96f449b85","Type":"ContainerDied","Data":"f43f73a5583a8dd16854de083dd514b2fddd3d738534479b1289752b9af6fefd"} Jul 06 20:08:43.848118 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.848071 2565 scope.go:117] "RemoveContainer" containerID="6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515" Jul 06 20:08:43.857230 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.857198 2565 scope.go:117] "RemoveContainer" containerID="6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515" Jul 06 20:08:43.857519 ip-10-0-143-100 kubenswrapper[2565]: E0706 20:08:43.857498 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515\": container with ID starting with 6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515 not found: ID does not exist" containerID="6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515" Jul 06 20:08:43.857585 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.857526 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515"} err="failed to get container status \"6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515\": rpc error: code = NotFound desc = could not find container \"6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515\": container with ID starting with 6ac10666b5bdd04ec5aa0a1e9ae42e19f3758db1ad35386c4ece24c837241515 not found: ID does not exist" Jul 06 20:08:43.871960 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.871931 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9"] Jul 06 20:08:43.877352 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:43.877330 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-04d79c-68658cb549-nzcv9"] Jul 06 20:08:45.204786 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:45.204749 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fe4e6d6-0278-45cd-bb4b-49f96f449b85" path="/var/lib/kubelet/pods/3fe4e6d6-0278-45cd-bb4b-49f96f449b85/volumes" Jul 06 20:08:46.419880 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:46.419839 2565 patch_prober.go:28] interesting pod/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 20:08:46.420266 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:46.419898 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:08:46.867484 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:46.867455 2565 generic.go:356] "Generic (PLEG): container finished" podID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerID="ad48145e676da840778b5e0faa713f0d1b102e93cab5f2c7852e155825250522" exitCode=0 Jul 06 20:08:46.867646 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:46.867497 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" event={"ID":"beec3cb2-c49d-48a9-bedb-49a7c696cfa4","Type":"ContainerDied","Data":"ad48145e676da840778b5e0faa713f0d1b102e93cab5f2c7852e155825250522"} Jul 06 20:08:46.867646 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:46.867565 2565 scope.go:117] "RemoveContainer" containerID="7d8aff1b6ee5626e7a034e96886afedb276f2e1e61e6381060ebaf8a16ac9f58" Jul 06 20:08:46.929602 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:46.929577 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:08:47.025334 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025245 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-credential-socket\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025334 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025323 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-data\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025546 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025349 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istiod-ca-cert\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025546 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025377 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-certs\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025546 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025411 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz7nr\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-kube-api-access-lz7nr\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025670 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025545 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-socket\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025670 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025559 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:08:47.025670 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025662 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-podinfo\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025823 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025700 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-token\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025823 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025752 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-envoy\") pod \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\" (UID: \"beec3cb2-c49d-48a9-bedb-49a7c696cfa4\") " Jul 06 20:08:47.025823 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025797 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:08:47.025968 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025876 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:08:47.025968 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.025887 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:08:47.026081 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.026060 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.026140 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.026082 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-credential-socket\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.026140 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.026096 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istiod-ca-cert\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.026140 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.026109 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-workload-certs\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.026331 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.026165 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-data" (OuterVolumeSpecName: "istio-data") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:08:47.027850 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.027818 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-kube-api-access-lz7nr" (OuterVolumeSpecName: "kube-api-access-lz7nr") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "kube-api-access-lz7nr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:08:47.028002 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.027821 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-token" (OuterVolumeSpecName: "istio-token") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:08:47.028258 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.028234 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:08:47.028354 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.028235 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "beec3cb2-c49d-48a9-bedb-49a7c696cfa4" (UID: "beec3cb2-c49d-48a9-bedb-49a7c696cfa4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:08:47.127149 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.127111 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-podinfo\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.127149 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.127144 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-token\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.127149 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.127157 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-envoy\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.127422 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.127166 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-istio-data\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.127422 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.127175 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz7nr\" (UniqueName: \"kubernetes.io/projected/beec3cb2-c49d-48a9-bedb-49a7c696cfa4-kube-api-access-lz7nr\") on node \"ip-10-0-143-100.ec2.internal\" DevicePath \"\"" Jul 06 20:08:47.873667 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.873627 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" event={"ID":"beec3cb2-c49d-48a9-bedb-49a7c696cfa4","Type":"ContainerDied","Data":"1c51d9c15dbde320a3b20e14d31f1feedb05463be36e174e6940624680826dc2"} Jul 06 20:08:47.873667 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.873650 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc" Jul 06 20:08:47.874245 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.873700 2565 scope.go:117] "RemoveContainer" containerID="ad48145e676da840778b5e0faa713f0d1b102e93cab5f2c7852e155825250522" Jul 06 20:08:47.899254 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.899196 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc"] Jul 06 20:08:47.909201 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:47.909171 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-cd90ec-openshift-default-769cdf7c67-4srbc"] Jul 06 20:08:49.204635 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:49.204605 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" path="/var/lib/kubelet/pods/beec3cb2-c49d-48a9-bedb-49a7c696cfa4/volumes" Jul 06 20:08:55.944145 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:55.944110 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6c4669c765-jgq9q_b1533c31-2ce6-4c55-bfd0-c993217553c8/maas-api/0.log" Jul 06 20:08:56.869061 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:56.869027 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586496c87-bnjhx_d43247a6-92b9-450f-a3cc-fdcdeec48774/manager/1.log" Jul 06 20:08:56.877369 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:56.877343 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586496c87-bnjhx_d43247a6-92b9-450f-a3cc-fdcdeec48774/copy-manifests/0.log" Jul 06 20:08:59.899166 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:08:59.899135 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bjpfr_ab90aa20-9137-4db0-9de5-31320ef227a1/istio-proxy/0.log" Jul 06 20:09:00.040547 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:00.040507 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hcnsl_aff7b4b2-5696-41d6-896c-4925c2e3f554/discovery/0.log" Jul 06 20:09:00.415588 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:00.415555 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-6pwvc_e91b4320-ae9e-4a37-b77e-dd445b836880/istio-proxy/1.log" Jul 06 20:09:00.666444 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:00.666358 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-hmg7z_c5bea8bb-2aba-4ba2-b5dd-663f11695f6d/payload-processing/0.log" Jul 06 20:09:01.413113 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:01.413080 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-gqz7n_d5478f16-fcbf-487a-ae7d-fc25120c37f8/main/0.log" Jul 06 20:09:01.651612 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:01.651582 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-zk6bm_b62fd00c-4f13-4185-bd7c-eb85a44ec894/main/0.log" Jul 06 20:09:09.532390 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:09.532356 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b6ks2_17bca18e-95ba-4d4b-b844-7f7b52c769f2/global-pull-secret-syncer/0.log" Jul 06 20:09:09.750636 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:09.750554 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wk8mp_7a726848-72dc-4069-965a-d61c7f72a321/konnectivity-agent/0.log" Jul 06 20:09:09.833702 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:09.833671 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-100.ec2.internal_0903f6f852d654ad4e05c5ebda33a529/haproxy/0.log" Jul 06 20:09:17.535394 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:17.535308 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-nct7w_cf9471e6-bb11-4bdf-b34e-f2821b1551a7/cluster-monitoring-operator/0.log" Jul 06 20:09:17.696088 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:17.696056 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-kmjwv_1ed35988-0b83-46eb-ae76-40c4e5857726/monitoring-plugin/0.log" Jul 06 20:09:17.827413 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:17.827385 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rr2rs_7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0/node-exporter/0.log" Jul 06 20:09:17.863141 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:17.863113 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rr2rs_7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0/kube-rbac-proxy/0.log" Jul 06 20:09:17.894197 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:17.894170 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rr2rs_7f85b68f-47c6-47f6-be2d-6f04f6e8f0d0/init-textfile/0.log" Jul 06 20:09:18.143409 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:18.143323 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8df2fa50-9067-4ed0-93bc-7743cacb7424/prometheus/0.log" Jul 06 20:09:18.169784 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:18.169746 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8df2fa50-9067-4ed0-93bc-7743cacb7424/config-reloader/0.log" Jul 06 20:09:18.197967 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:18.197933 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8df2fa50-9067-4ed0-93bc-7743cacb7424/thanos-sidecar/0.log" Jul 06 20:09:18.225086 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:18.225054 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8df2fa50-9067-4ed0-93bc-7743cacb7424/kube-rbac-proxy-web/0.log" Jul 06 20:09:18.253412 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:18.253387 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8df2fa50-9067-4ed0-93bc-7743cacb7424/kube-rbac-proxy/0.log" Jul 06 20:09:18.285725 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:18.285696 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8df2fa50-9067-4ed0-93bc-7743cacb7424/kube-rbac-proxy-thanos/0.log" Jul 06 20:09:18.313806 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:18.313775 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8df2fa50-9067-4ed0-93bc-7743cacb7424/init-config-reloader/0.log" Jul 06 20:09:18.406946 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:18.406863 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-g6659_d24b55c0-3da1-4638-a4a2-f4a0df59fbcd/prometheus-operator-admission-webhook/0.log" Jul 06 20:09:19.364897 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.364865 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n"] Jul 06 20:09:19.365296 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365265 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" Jul 06 20:09:19.365296 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365278 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" Jul 06 20:09:19.365296 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365285 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7a25b2-8879-4072-83a3-779637d0963b" containerName="maas-api" Jul 06 20:09:19.365296 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365291 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7a25b2-8879-4072-83a3-779637d0963b" containerName="maas-api" Jul 06 20:09:19.365296 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365297 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe4e6d6-0278-45cd-bb4b-49f96f449b85" containerName="maas-api" Jul 06 20:09:19.365492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365302 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe4e6d6-0278-45cd-bb4b-49f96f449b85" containerName="maas-api" Jul 06 20:09:19.365492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365315 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" Jul 06 20:09:19.365492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365321 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" Jul 06 20:09:19.365492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365383 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7a25b2-8879-4072-83a3-779637d0963b" containerName="maas-api" Jul 06 20:09:19.365492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365390 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="beec3cb2-c49d-48a9-bedb-49a7c696cfa4" containerName="istio-proxy" Jul 06 20:09:19.365492 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.365396 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fe4e6d6-0278-45cd-bb4b-49f96f449b85" containerName="maas-api" Jul 06 20:09:19.368430 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.368414 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.371223 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.371186 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-szkqh\"/\"openshift-service-ca.crt\"" Jul 06 20:09:19.371345 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.371256 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-szkqh\"/\"kube-root-ca.crt\"" Jul 06 20:09:19.371411 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.371370 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-szkqh\"/\"default-dockercfg-bgf4z\"" Jul 06 20:09:19.378603 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.378575 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n"] Jul 06 20:09:19.527291 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.527246 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-proc\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.527291 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.527299 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w97lh\" (UniqueName: \"kubernetes.io/projected/d70f5877-20c6-411b-ad99-e0c01bb04c56-kube-api-access-w97lh\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.527536 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.527358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-sys\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.527536 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.527386 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-podres\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.527536 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.527412 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-lib-modules\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628641 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628538 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-proc\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628641 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628589 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w97lh\" (UniqueName: \"kubernetes.io/projected/d70f5877-20c6-411b-ad99-e0c01bb04c56-kube-api-access-w97lh\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628641 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628639 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-sys\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628659 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-proc\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-podres\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628727 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-lib-modules\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628747 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-sys\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628793 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-podres\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.628904 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.628867 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d70f5877-20c6-411b-ad99-e0c01bb04c56-lib-modules\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.638561 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.638533 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w97lh\" (UniqueName: \"kubernetes.io/projected/d70f5877-20c6-411b-ad99-e0c01bb04c56-kube-api-access-w97lh\") pod \"perf-node-gather-daemonset-mzw9n\" (UID: \"d70f5877-20c6-411b-ad99-e0c01bb04c56\") " pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.678686 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.678656 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:19.814629 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:19.814595 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n"] Jul 06 20:09:19.817699 ip-10-0-143-100 kubenswrapper[2565]: W0706 20:09:19.817665 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd70f5877_20c6_411b_ad99_e0c01bb04c56.slice/crio-cde20fb68ab61bada5731a9987b8350bf455cc16bd1fdf2904acfe793d54bd8a WatchSource:0}: Error finding container cde20fb68ab61bada5731a9987b8350bf455cc16bd1fdf2904acfe793d54bd8a: Status 404 returned error can't find the container with id cde20fb68ab61bada5731a9987b8350bf455cc16bd1fdf2904acfe793d54bd8a Jul 06 20:09:20.021688 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:20.021654 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" event={"ID":"d70f5877-20c6-411b-ad99-e0c01bb04c56","Type":"ContainerStarted","Data":"0a6fe4c61066bbe5ae7df5b958ff10c97c0077a4f13338acbf731a01b233417b"} Jul 06 20:09:20.021879 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:20.021707 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" event={"ID":"d70f5877-20c6-411b-ad99-e0c01bb04c56","Type":"ContainerStarted","Data":"cde20fb68ab61bada5731a9987b8350bf455cc16bd1fdf2904acfe793d54bd8a"} Jul 06 20:09:20.021879 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:20.021768 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:20.574579 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:20.574545 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/1.log" Jul 06 20:09:20.584760 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:20.584732 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-6zvvd_6f168e98-d917-4b91-9dfd-1a47192a428d/console-operator/2.log" Jul 06 20:09:23.103282 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:23.103252 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gggzl_9efbb99a-2d00-47b8-9188-4a35489cff63/dns/0.log" Jul 06 20:09:23.127626 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:23.127598 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gggzl_9efbb99a-2d00-47b8-9188-4a35489cff63/kube-rbac-proxy/0.log" Jul 06 20:09:23.241314 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:23.241283 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-blvmv_d1df10a1-d344-41da-a87f-1df534ffc631/dns-node-resolver/0.log" Jul 06 20:09:23.891472 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:23.891438 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8447b76dd-grsn2_e8c5b066-7aa9-487a-8dd1-54ce40af48c8/registry/0.log" Jul 06 20:09:23.924069 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:23.924039 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pdr9_77d302df-429b-47c4-a97c-c8097c1db9fd/node-ca/0.log" Jul 06 20:09:25.084138 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:25.084111 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bjpfr_ab90aa20-9137-4db0-9de5-31320ef227a1/istio-proxy/0.log" Jul 06 20:09:25.211142 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:25.211114 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hcnsl_aff7b4b2-5696-41d6-896c-4925c2e3f554/discovery/0.log" Jul 06 20:09:25.310140 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:25.310109 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-6pwvc_e91b4320-ae9e-4a37-b77e-dd445b836880/istio-proxy/0.log" Jul 06 20:09:25.338453 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:25.338371 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-6pwvc_e91b4320-ae9e-4a37-b77e-dd445b836880/istio-proxy/1.log" Jul 06 20:09:25.411562 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:25.411532 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-hmg7z_c5bea8bb-2aba-4ba2-b5dd-663f11695f6d/payload-processing/0.log" Jul 06 20:09:26.035834 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:26.035801 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" Jul 06 20:09:26.057126 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:26.057069 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-szkqh/perf-node-gather-daemonset-mzw9n" podStartSLOduration=7.057050439 podStartE2EDuration="7.057050439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:20.040460722 +0000 UTC m=+2591.414916662" watchObservedRunningTime="2026-07-06 20:09:26.057050439 +0000 UTC m=+2597.431506387" Jul 06 20:09:26.309431 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:26.309347 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gflxp_a7d93137-2531-41a7-9523-a824b57b3dae/serve-healthcheck-canary/0.log" Jul 06 20:09:27.042233 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:27.042194 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hbh7b_c3d34406-18c8-4adf-ba4b-4ed16cb0e367/kube-rbac-proxy/0.log" Jul 06 20:09:27.071533 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:27.071512 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hbh7b_c3d34406-18c8-4adf-ba4b-4ed16cb0e367/exporter/0.log" Jul 06 20:09:27.105995 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:27.105965 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hbh7b_c3d34406-18c8-4adf-ba4b-4ed16cb0e367/extractor/0.log" Jul 06 20:09:29.611593 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:29.611569 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bjpfr_ab90aa20-9137-4db0-9de5-31320ef227a1/istio-proxy/0.log" Jul 06 20:09:29.719514 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:29.719483 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hcnsl_aff7b4b2-5696-41d6-896c-4925c2e3f554/discovery/0.log" Jul 06 20:09:29.769776 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:29.769746 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-6pwvc_e91b4320-ae9e-4a37-b77e-dd445b836880/istio-proxy/0.log" Jul 06 20:09:29.807228 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:29.807183 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-6pwvc_e91b4320-ae9e-4a37-b77e-dd445b836880/istio-proxy/1.log" Jul 06 20:09:29.871089 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:29.870993 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-hmg7z_c5bea8bb-2aba-4ba2-b5dd-663f11695f6d/payload-processing/0.log" Jul 06 20:09:29.987063 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:29.987029 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6c4669c765-jgq9q_b1533c31-2ce6-4c55-bfd0-c993217553c8/maas-api/0.log" Jul 06 20:09:30.439255 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:30.439204 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586496c87-bnjhx_d43247a6-92b9-450f-a3cc-fdcdeec48774/manager/1.log" Jul 06 20:09:30.446599 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:30.446576 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586496c87-bnjhx_d43247a6-92b9-450f-a3cc-fdcdeec48774/manager/0.log" Jul 06 20:09:30.466971 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:30.466946 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586496c87-bnjhx_d43247a6-92b9-450f-a3cc-fdcdeec48774/copy-manifests/0.log" Jul 06 20:09:31.118687 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:31.118653 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bjpfr_ab90aa20-9137-4db0-9de5-31320ef227a1/istio-proxy/0.log" Jul 06 20:09:31.238021 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:31.237937 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hcnsl_aff7b4b2-5696-41d6-896c-4925c2e3f554/discovery/0.log" Jul 06 20:09:31.291046 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:31.291015 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-6pwvc_e91b4320-ae9e-4a37-b77e-dd445b836880/istio-proxy/0.log" Jul 06 20:09:31.322593 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:31.322563 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-6pwvc_e91b4320-ae9e-4a37-b77e-dd445b836880/istio-proxy/1.log" Jul 06 20:09:31.373977 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:31.373890 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-hmg7z_c5bea8bb-2aba-4ba2-b5dd-663f11695f6d/payload-processing/0.log" Jul 06 20:09:32.066880 ip-10-0-143-100 kubenswrapper[2565]: I0706 20:09:32.066852 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-94489db4c-rnglb_f557ac52-d837-47c1-931b-8b48f7462122/manager/0.log"